Does peer review work?

Peer review is the process by which journals evaluate articles submitted to them, to decide whether to publish them. It involves the commenting on the papers by anonymous "peers" - that is, others in the field. It is generally regarded as an integral part of the process of publication and is respected by many - but is it the right way to go?

I ask for a reason. Sometimes peers are not as insightful as one might suppose. Their views may be unfounded, misconceived or simply plain wrong - yet they determine whether a particular paper is published. A recent example comes to mind. I submitted a paper to a journal and received comments in due course. What I found interesting, was the difference in approach of the reviewers. One was positive in his examination of the paper, looking at the good and suggesting ways it could be even better. He thought the paper very interesting. The other reviewer, however, was extremely aggressive and seemed personally affronted by the paper. What was very telling about his review was that he had MISUNDERSTOOD THE PAPER. His critiques were based on perceived flaws that had no relevance to the paper. He even criticized it because he wrote that something was not included. In fact, it was included and was clearly stated early on in the paper. It was clear that he had not even read the paper with any care, at all.

Guess which reviewer the editor had been more influenced by? That is right: the negative review. Yet, the negative review was plain wrong in his understanding of the paper. His criticisms, though phrased in an erudite manner, were erroneous to the core. They showed that he did not understand the experiment, or why it had been conducted in the manner it had. Yet, rather than see through the problems with this reviewers review, the editor had, it is clear, been convinced by it. The other quite positive review was ignored in its favour.

Thus it is that the reviewer who saw value in the paper was ignored and the reviewer who did not understand the paper, won out. Clearly, this is not how it should be. Reviewers determine the fate of papers, yet reviewers may not be insightful, knowledgeable, impartial or understanding. Sometimes, reviewers write responses for reasons that have nothing to do with the contents of the paper itself. For instance, the negative reviewer seemed to take exception to my co-author, from comments he made, that, perhaps, he did not realize I would see.

I think that peer reviews should not be allowed to impede the dissemination of scientific work. They should take place after publication and not before. They should be public, for all to see - and perhaps it would be better if they were not anonymous, so that certain reviewers would not be free to pursue private agendas, in secret.

All papers should be publishable at once. Then, the assessment process should begin. The system, as is, lends itself to abuse. It is too easy to suppress another's work, to frustrate another's progress, if that is one's intent. It is also rather random whether or not reviewers are good - that is, whether they understand the work and are true to it - or not.

Personally, I found it rather odd that one reviewer should have expressed himself with such forceful hostility - and yet, the other reviewer clearly appreciated the work and was of a mind to let it be published, with some well chosen improvements. It was almost as if they had been reading different papers. (Except for the fact that the hostile reviewer hadn't read it carefully at all).

There are moves, in some quarters, towards changing the way peer review is done. There are some places where peer review takes place after publication. This is the best way - from the point of view of access to knowledge. I hope to see it become the predominant paradigm, in years to come. After all, for most of the history of science, peer review just was not done at all - and that didn't stop science from advancing. Einstein's four famous papers in his "annus mirabilis", of 1905, were not peer reviewed. They were simply published because the editor of the journal, in question, thought them interesting. Perhaps, today, he would have had problems publishing them at all, facing reviewers of the kind I encountered with my own paper: ones who did not understand the work in front of them.

History shows that scientific advancement is not dependent on peer review (which only really became dominant from the middle of the last century)...so why is academia so enamoured of it? Could it be that peer review, itself, needs to be reviewed?

(If you would like to learn more of Ainan Celeste Cawley, 10, or his gifted brothers, Fintan, 6 and Tiarnan, 4, this month, please go to:
http://scientific-child-prodigy.blogspot.com/2006/10/scientific-child-prodigy-guide.html

I also write of gifted education, child prodigy, child genius, adult genius, savant, megasavant, HELP University College, the Irish, the Malays, Singapore, Malaysia, IQ, intelligence and creativity.

My Internet Movie Database listing is at: http://imdb.com/name/nm3438598/
Ainan's IMDB listing is at http://imdb.com/name/nm3305973/
Syahidah's IMDB listing is at http://imdb.com/name/nm3463926/

Our editing, proofreading and copywriting company, Genghis Can, is at http://www.genghiscan.com/

This blog is copyright Valentine Cawley. Unauthorized duplication is prohibited. Use only with permission. Thank you.)