May 312016
 

The Publishing Research Consortium has just released the results of a 2015 survey on peer review for which I did the analysis and report. From the press release:

PRC Report Reveals: Broad Support for Peer Review, Desire for Improvements Increasing

Peer review still broadly supported, continuing preference for conventional, pre-publication, single or double blind peer review

London, June 2016: The Publishing Research Consortium has commissioned STM publishing consultant Mark Ware to analyse and report on data from a new survey of opinions and attitudes to peer review among research authors and reviewers.

Overall, the report reveals that peer review remains the principal means to establish trust in the scholarly literature, and is the filtering mechanism that sustains quality assurance. Despite a continuing high level of support for peer review, as the practices of the academic community evolve so does a healthy scrutiny of the system, with its effectiveness, fairness, sustainability, and cost-effectiveness all subject to challenge and debate.

The report concludes:

  • Satisfaction with and broad support for peer review has remained remarkably stable from previous surveys, but the desire to see improvements in specific areas is increasing.
  • The effectiveness of peer review is ranked highest for improving the quality of the published paper, but researchers also believe that peer review should be able to detect fraud and plagiarism.
  • The Survey shows a clear continuing preference for conventional, pre-publication, single or double blind peer review, both as authors and reviewers, with open peer review ranking significantly behind, although there is some variability between subject communities on the topic of alternative types of peer review.
  • The burden on peer reviewers remains significant. Participating in peer review is, however, seen as an important contribution to the community and reciprocating the work of others.
  • Overall, satisfaction or dissatisfaction with peer review does not vary predictably with demographic factors like geographic region, subject discipline or age/seniority of researcher. That is, there is no simple pattern of support, and in particular, criticism of peer review is not significantly greater among younger researchers or those from non-Western countries.

Mark Ware comments: “Although the overall picture has evolved somewhat since my survey for PRC in 2007, support for peer review has remained remarkably stable. However there are tensions to resolve and clearly further evolution in the system is required. On the one hand researchers want peer review to happen faster, but they also value high quality, constructive review. They want to reduce the possibility of bias or fraud in the system, but there is still no consensus on how best to deliver that.”

Michael Mabe, Chair of PRC comments: “Peer review is the critical element that sets the scholarly literature apart from mere discourse. The system merits continuing challenge and debate, and PRC can contribute evidence for that debate by recording the evolution of attitudes and opinions over time. We hope this Report will be received as a further effective contribution to the literature of peer review.”

The survey instrument was constructed to enable longitudinal comparison with two previous surveys: by Mark Ware for PRC in 2007 and by Sense About Science in 2009. Ware also compares his conclusions from a third survey by Taylor & Francis in 2015, which is a valuable complement to this PRC study.

To add context and depth to a quantitative analysis, the Report also includes many verbatim (free text) responses, chosen to be consistent with the overall views of the whole sample but offering some suggestion of where the system might be improved.

The Publishing Research Consortium Peer Review Survey 2015 is available from publishingresearchconsortium.com, with a CC-BY-NC-ND licence.

The Publishing Research Consortium is a group of associations and publishers that support research into global issues that impact scholarly communication, in order to promote evidence-based discussion. Our steering group comprises representatives from the International Association of STM Publishers, The Publishers Association, Association of Learned and Professional Society Publishers, Association of American Publishers, Taylor & Francis, Elsevier, Springer Nature, and Wiley.