Updating Peer Review

The post was contributed by Robin N. Sinn at Johns Hopkins University. It originally appeared in the Sheridan Libraries Blog.

Peer reviewed journals are the bedrock of the scholarly publishing system, ensuring that an article’s authors have used proper methods, cited previous work appropriately, and made logical and supported conclusions. The process of peer review is changing for several reasons:

In fact, this month will see the second annual Peer Review Week. This year’s theme is Recognition for Review and runs from September 19th through 25th. A recent Scholarly Kitchen post asked the Chefs about the future of peer review.

Below are a few of the groups trying to improve peer review.  In October 2015 ASERL hosted a webinar with representatives from three of these groups. You can watch the session recording or look at the slides from Peerage of Science, eLife, and Rubriq.

Shortening the Traditional Process

There is a concern that the peer review process takes too long. An editor makes a decision to send the article out for review, finds the reviewers, the review happens, comments are gathered and sent to the author, revision happens, resubmission… you get the picture. Lots of time can pass. A few groups are tightening up that process.

  • PLoS ONE was the first of a new kind of mega-journal that aims to publish articles that are methodologically and scientifically sound. Time is not spent on analyzing the importance of the article or the fit between journal and article. This cuts out the first part of the review process.
  • eLife shortens the review process by compiling revision requests from reviewers into one document and having only one reviewer examine resubmitted papers.

Peer Review Independent of Specific Journal

Instead of each journal wrangling their own set of peer reviewers and reviewing papers multiple times as they bounce around the system, a few groups are providing peer reviews that can be used by any journal.

  • At Rubriq the author pays for a review, then receives a report from 3 reviewers along with journal recommendations. The author can then revise the manuscript (or not) and submit the manuscript to a journal of their choice, including the Rubriq report as supplemental material if they wish.
  • Peerage of Science is supported by journals subscribing to their services. Reviewers have certain criteria to meet when they make their reviews, so their reviews are reviewed. This gives authors and journals a way to rate reviewers.  Once the reviews are done, articles are available to subscribing journals. Authors are able to make the reviews available to non-subscribing journals.

Post-Publication Peer Review

These groups post articles after they pass a set of minimum criteria. The peer review takes place online, in full view of readers.

Credit for Peer Reviewers

With the increasing number of research articles and journals available, there’s an increasing need for peer reviewers. Given that researchers spend their own time reading and reviewing, there’s an interest in giving peer reviewers credit for their work.

  • Publons assigns points for writing reviews of articles that are published. The reviews can be published, dependent on journal rules. Reviewers who write the most reviews receive awards and certificates. The idea is to ‘reward’ reviewers so that they do better work.
  • Other groups (F1000Research, ScienceOpen, among others) are giving peer reviewers the opportunity to sign their reviews, thus breaking the tradition of anonymous peer review.

I’m sure there are other peer review experimenters out there. If you know of one, please share with the ASERL SCIG at aserlscholcomm@aserl-lists.org .