Thursday, September 22, 2016

Tastes Like the Real Thing

And in the category of "peer review is so screwed", some researchers machine-generated reviews and presented them along with actual reviews to participants, who generally could not notice a difference:
Peer review is widely viewed as an essential step for ensuring scientific quality of a work and is a cornerstone of scholarly publishing. On the other hand, the actors involved in the publishing process are often driven by incentives which may, and increasingly do, undermine the quality of published work, especially in the presence of unethical conduits.

We presented to [16] subjects a mix of genuine and machine generated reviews and we measured the ability of our proposal to actually deceive subjects judgment. The results highlight the ability of our method to produce reviews that often look credible and may subvert the decision.
God help us all.

No comments:

Post a Comment