Is peer review bad for science?

By David Wojick @ CFACT 

After studying the popular practice of peer review of scientific journal articles for several years, I have reluctantly concluded that peer review is bad for science. While the practice has its good side, there are several ways that it greatly impedes progress, and the bad greatly outweighs the good.

To begin with, let’s look at what peer review tries to do. The obvious thing is to block the publication of fake science. However this appears to be a rare event in most sciences. There are several million journal articles published each year, all peer reviewed, typically by two or three reviewers. Clearly these many millions of reviews did not keep any of these myriad articles from being published.

Paradoxically, however, most of these articles were in fact rejected based on peer review; many were rejected many times. Top journals often boast of having high rejection rates, like 80% or so. If this is the general practice then the average article must be submitted to something like five journals before it is accepted and published. If each submission is peer reviewed then that is a lot of reviews per article, perhaps ten to fifteen on average.

Given that all of these multiply rejected articles eventually get published, something other than simple gate keeping must be going on. This something looks to be an extremely laborious sorting process, whereby each article eventually finds the “right” journal. It is hard to see any value being added by these many millions of peer reviews. Given modern search technologies, which journal an article ultimately appears in no longer seems very important.

One negative aspect of peer review is well known. This is where gate keeping keeps great new ideas from being published. Max Planck, who discovered the quantum nature of energy, put it very nicely, saying something like “Your ideas will (only) be accepted when your students become journal editors.” This is the dark side of peer review blocking science, the novel good ideas get blocked as bad ideas.

But there are several other bad things that flow from peer review that I have not seen mentioned. These down sides are features of the incredibly time consuming and laborious nature of the practice.

First there is the huge time delay between the time a paper is written and when it is finally published. Let’s say that peer review takes four months, which is probably pretty fast. If the average paper is reviewed five times then that is almost two years of reviews before it is finally accepted. (Also, there are many other steps between these reviews, so the average might be more like four years from first submission to final publication.)

If two million papers are published each year, with an average delay of say two years each, due to peer review, that is an accumulation of four million years of delay every year. It is reasonable to believe that eliminating this vast tide of delay would dramatically speed up the progress of science.

Then there is the cost. Organizing and managing the peer review process is probably the greatest expense that journal publishers face. Keep in mind that given an 80% rejection rate, something like five articles will be reviewed for every one published. At three reviews each that means fifteen reviews per published articles.

The high cost of journals and articles is a major obstacle to access by all but the richest universities and researchers. This to probably greatly impedes the progress of science.

Then there is the huge amount of time that researchers spend reviewing each other’s articles. Reviews are expected to be comprehensive, so they probably take from 10 to 20 hours each, maybe more. If there are fifteen reviews per article published that is 150 to 300 hours of review time.

Multiply that by 2 million articles published and we get an incredible 300 to 600 million hours a year devoted to reviewing, rather that to research. Assuming that a work year is 2000 hours, this is like taking 150 to 300 thousand researchers off the job, just to peer review each other’s papers. Think of what that amount of research might create. Again, this is a huge loss to the progress of science.

Conclusion: Peer review adds an enormous amount of delay, cost and distraction to the process of science. It does not do enough good to justify these huge adverse impacts on the rate of scientific progress. Thus on balance peer review is bad for science. Author