Editorial for Publications March 2016

Welcome to the latest issue of Publications, a journal for studies and opinion on all aspects of scholarly publishing and communication. A specific welcome from me too, as the new Editor-in-Chief.

Our journal is fully “peer reviewed” or “refereed”. These expressions are synonymous although, despite modest research, I’ve not discovered the origin of the word “referee” in this context—if any of you know or think you know, do tell me ([email protected]). In the publishing houses within which I’ve worked, we’ve always called them “referees” and had “referee databases”. But “peer review” is certainly more fashionable, and descriptive.

In any event, I just wanted to say a word or two about it. Feeling quite active again in publishing as a non-exec “Board Manager” for the American Institute of Physics Publishing, over the last year or so I’ve, once more, got out of the way of keeping up with general literature and discussion on such topics (I left the editorship of Learned Publishing at the end of 2014). So it was with mixed emotions that I read the interview with Jeffrey Beall on the Scholarly Kitchen [1] and the, in fact, more interesting following comments and discussion (despite the somewhat patronizing tone of some of them). On the one hand, I was pleased that I’ve not missed anything—nothing has changed in that discussion. But I also found all those all tired (even if correct) arguments a little depressing. What stops us moving on? Mustering considerable hubris, I think I know. The problem is that almost all the comments are correct, in one way or another, they are just not sufficiently correct. They are often, if not orthogonal, then tangential to each other—and they all seem to miss, even if some of the authors implicitly know, a few points about the whole system. Of course, now here comes the wisdom to put them all straight.

There are a few misunderstandings about peer review that are somehow perpetuated even when the perpetrators know or should know better. Two important ones are:

(i)

Peer review is an on/off or binary switch, i.e., it’s about acceptance/rejection. No it isn’t—well, not only that. It’s about review and improvement too. I know for a fact (you will have to ask me personally if you want to know how I know this) that for hundreds of thousands of stm articles published over a year, something like 90% are revised after review and before publication. Sometimes with major revisions, sometimes minor, but you would have to be a monumental cynic not to believe that that results in some significant improvement to the published output. That percentage may even underestimate the level of revision, since, as we know, rejected items, often can then be revised as they are submitted to another, possibly lower-ranked journal, where the first revision will not be recorded. Of course, we are talking here about review administered by the publication outlet. Most sensible researchers, especially junior ones, will have colleagues, mostly likely in their own department, informally review the paper even before submission.

(ii)

Acceptance means the reviewers/editor think it’s “right”, and rejection means they think it’s “wrong”. No, it doesn't mean that. “Acceptance” means, or should mean, that they think it is publishable, and publishable in that journal. No more than that. Of course, you hope that peer review will pick up egregious, or even minor, errors. But rarely, except perhaps in some highly theoretical or math topics, could a reviewer be 100% certain of any pronouncement. A lot must be taken on trust, and even where, say, an experimental procedure is reproducible or replicable, a reviewer is not expected to do it (that’s for other researchers, if they wish and can get a grant). One of the best quotes I saw in a referee report was along these lines: ‘This is remarkable. I don’t think it’s correct, but I can’t fault the science as presented. So it deserves to be published so that others can comment and try it for themselves’. That’s a pretty good encapsulation of the scientific or scholarly process.

In fact, the last point is a key one—the process of science (as distinct from science itself) is a social activity mediated, at its best, by shared norms and behaviours. Having work reviewed by peers and, just as important, reviewing and influencing the work of others, are key parts of the scholarly process. Being a reviewer is a key part of becoming a scientist. It also explains why, for the most part, post publication peer review does not much interest scholars nor scientists. There are two explicit reasons. Many will worry that publicly commenting on published work can seem a little self-aggrandizing (that will not put off all of them of course!); more importantly, when they are asked, they would much prefer having the chance to comment (anonymously or not) before publication so that they can affect and hopefully improve what actually is published.

We all know peer review is an imperfect system, with demonstrable instances of bias, mistakes and perversion. But we’re hard pressed to think of a better system. We have already published some interesting items on peer review in this journal. There will be more. Neither peer review nor the debate on it are going to go away any time soon.

One of the great things about editorials is that the editor can pontificate without having to debate—luckily editorials from me are not peer reviewed. But if you have an opinion on this topic, or anything else, and can argue it well, and would like to see it in Publications, do get in touch at the email mentioned above. I look forward to hearing from you.