Last year the Nature open peer review trial found that
… A small majority of those authors who did participate received comments, but typically very few, despite significant web traffic. Most comments were not technically substantive. Feedback suggests that there is a marked reluctance among researchers to offer open comments.
It seems that researchers are just not yet ready to use these kinds of comment facilities. Further evidence comes from PLoS ONE, the open access online journal of everything from the Public Library of Science. PLoS ONE has a deliberately low barrier to publication in the form of minimal peer review and intends that the better work will float to the surface through users’ comments and ratings. But despite PLoS ONE having been surprisingly successful at attracting authors (given that authors have to pay $1250 publication charge for a journal with no impact factor and low peer review standards), it has been much less successful at getting users to comment on articles.
PLoS ONE has therefore hired a well-known science blogger, Bora Zivkovic as Online Community Manager with the specific responsibility for drumming up comments. Bora has recently published a FAQ on the PLoS blog for PLoS ONE users thinking of commenting, which includes this extract:
Q: I think the article has a major problem, but I am afraid to challenge a big name in my field.
Your nervousness is understandable. But, if you believe that you have identified a real problem with the article and you feel confident about it, it is likely that other readers will feel the same. Be the first one to comment about it (try to use non-confrontational language such as ‘could’ not ‘should’ etc) and read the responses of others who may agree or disagree with you. On PLoS ONE everyone is equal and everyone is expected to treat others with equal respect. Courage to challenge authorities will gain you a fair reputation among your peers.
So, “Courage to challenge authorities will gain you a fair reputation among your peers.” Do I hear the sound of hollow laughter …?
On the other hand, the Rapid Response feature of the British Medical Journal is very popular (so much so that editorial moderation had to be tightened to curtail the “bores who monopolised conversations for compelling personal reasons”). What’s the difference? It may be that the BMJ Rapid Responses were seen as simply an electronic version of an already existing (and popular) feature, the Letter to the Editor, whereas the Nature and PLoS One examples were wholly or partly attempts to develop new forms of peer review. Despite the many well-known problems with peer review, it seems the scientific community is extremely conservative about changes to it.