Peer Review Transparency and Scientific Journalism

The folks over at Nature Communications recently announced that peer reviews will have the option of being published alongside the accepted manuscript. The author(s) will get to decide whether they are published, and reviewers will have the option to remain anonymous or have their identities revealed.

By enabling the publication of these peer review documents, we take a step towards opening up our editorial decision-making process. We hope that this will contribute to the scientific appreciation of the papers that we publish, in the same way as these results are already discussed at scientific conferences. We also hope that publication of the reviewer reports provides more credit to the work of our reviewers, as their assessment of a paper will now reach a much larger audience.

I wonder, though, what impact this decision is liable to have on the peer review process. As I've blogged before (here and here), there are certainly benefits and drawbacks to peer review. Though the editor has the final decision of whether to publish a paper, peer reviewers can provide important feedback the editor may not have considered, and can be selected for their expertise in the topic under study (expertise the editor him/herself may not have). At the same time, reviewer comments are not always helpful, accurate, or even professional in their tone and wording.

The question, then, is whether peer review transparency will change any of that. Will reviewers be on better behavior if they think their comments may be made public? Perhaps, though they can still remain anonymous. One hopes it will at least encourage them to be more clear and descriptive with their comments, taking the time to show their thought process about a certain paper.

Of course, the impact of peer review transparency wouldn't just stop there. How might these public reviews be used by others? One potential issue is when news sources cover scientific findings. When I used to teach Research Methods in Psychology, one of my assignments was to find a news story about a study, then track down the original article, and make comparisons. Spoiler alert: many of the news stories ranged from being slightly misleading to completely inaccurate in their discussions of the findings. Misunderstanding of statistics, overestimating the impact of study findings, and applying findings to completely unrelated topics were just some of the issues.

Probably part of the issue is lack of scientific literacy, which is a widespread problem. There is no training for journalists covering research findings, though one wonders, if there were, whether it would look something like this:

Source: SMBC Comics (one of my favorites)

Journalists also tend to draw upon other sources when covering research findings, such as talking to other researchers not involved with the study. It seems likely that, if transparent peer review becomes a widespread thing, we can expect to see reviewer comments in news stories about research. If the reviewer has requested to remain anonymous, there would be no way to track down the original reviewer for clarification or additional comments, or even to find out the reviewer's area of expertise. And since reviews stop once the paper has been accepted - but revisions don't necessarily stop then, instead going through the editor - comments may not be completely "up-to-date" with the contents of the paper. So there seems to be some potential for misuse of these comments.

I'm not suggesting we hide this process. In fact, my past blog posts on peer review were really about doing the opposite. I just wonder what ramifications this decision will have on publishing. It should be noted that Nature Communications is planning to assess after a year whether this undertaking was successful. But if it is successful, is this something we think we'll see more of at other journals?

Transparently yours,
~Sara