Open Peer Review in OpenAIRE

OpenAIRE supports OA infrastructure in Europe and beyond, to help realize Open Science for the benefit of society, innovation and industry. OpenAIRE2020, the current project phase, is investigating a full complement of scholarly communication building blocks, including research data management support, gold open access, usage statistics, Linked Open Data, a publication broker and global interoperability. As part of this effort, Göttingen State and University Library, in partnership with COUPERIN in France and the Consiglio Nazionale Delle Ricerche (CNR) in Italy, will be conducting research into new peer review approaches.

What is open peer review?

[caption id="attachment_145" align="alignright" width="300"] Credit: AJ Cann, CC BY-SA 2.0


Informally, peer review is just the development of nascent ideas and theories through critical discussion with others. As such it is as old as science itself. Its stricter sense – the formal scholarly publishing process where an editor sends copies of a manuscript to people judged knowledgeable enough to be able to comment on its suitability for publication – is more recent. In this mode it has been the default of academic publishing since just the mid 20^th Century.

Peer review serves to validate the soundness, the substance and the originality of a work, or to help improve it until it meets these criteria. Peer review, in its current form, is generally:

• Anonymous: Either the author doesn’t know who the reviewer is (single-blind) or author and reviewer are both unknown to each other (double-blind).
• Selective: The reviewers are chosen by the editor.
• Hidden: The process takes place behind closed doors (or rather password privileges) and the reviews themselves are not published.

Open review (OR) is the name given to a variety of new review methods that remove one or more of these conditions. Hence, “open” can mean either a lack of anonymity, self-selecting reviewers, a public process or some mixture of the three.

What problems is OR trying to fix?

Studies have shown that although academics are, on the whole, fairly content with peer review, they think it could work better. Here are some of its problems:

• Time: Peer review often takes a long, long time. Could open review help speed up the process?
• Accountability: The anonymity of reviewing, although in principle meritocratic (as junior researchers can criticize the work of luminaries without fear of reprisals), also makes it unaccountable (some say Kafkaesque). Should professionals cast judgments in secret? Shouldn’t they be prepared to stand openly by what they believe? Relatedly, would reviews conducted in public be more constructive and less confrontational?
• Bias: Given the specialized nature of academia, a researcher’s nearest “peers” will very often be known to them as either friends or rivals. This fact naturally leads to concern that rejection or acceptance might sometimes have social, rather than scientific, grounds. Even where authors’ names are withheld, it is often clear from the research itself (or just searching Google) who the author is. If reviews were public, such biases might be further suppressed.
• Scams: That the peer process takes place behind closed doors also likely aids peer review scams to avoid detection.
• Incentive: Reviewing, done well, is hard work. If reviews were open, busy academics and researchers could take credit for them, demonstrating experience, community involvement and impact.
• Wasted effort: Reviewer comments often add context or point to areas for future work. Reviewer disagreements can expose areas of tension in a theory or argument. Readers may find such information helpful and yet at present, this potentially valuable additional information is wasted. Furthermore, as rejected papers are usually resubmitted elsewhere (to be reviewed all over again), there is a duplication of effort –the same paper might be reviewed multiple times.

Different kinds of OR could help address some of these problems. Of course, OR is no panacea and new solutions can bring new difficulties. Nonetheless, in the scope of the wider Open Science movement, it is worth asking if our new digital, networked technologies can help us improve peer review to make it better able to do what we want it to do: improve and select academic work such that it forms a sound and substantial addition to the literature or evidential base of a subject area.

Open Review and OpenAIRE

As part of its investigation of new modes of scholarly communication, OpenAIRE2020 will be looking at new modes of open peer review. This work will involve a landscape scan and evaluation of new forms of publishing and OR, considering various publishing routes and publication types. We’ll gather information via literature research and a multi-stakeholder survey (involving research communities, publishers, innovative projects, etc.), as well as build and evaluate prototype services. In the Social Sciences and Humanities, hypotheses.org, a platform hosting academic blogs, will carry out experiments to model a workflow that leads from blog articles - via selection, review and revision - to peer reviewed articles and books. We aim to practise what we preach and publish our results using OR methods and the facilities provided by the prototypes.

In order to encourage experimentation in OR, we’ll shortly be publishing a call for tenders for two prototypes in this area. This and other updates on project progress will be posted here. In the meantime, if you have any questions or comments please do contact us!