Google
 
Web www.scientific-child-prodigy.blogspot.com

The boy who knew too much: a child prodigy

This is the true story of scientific child prodigy, and former baby genius, Ainan Celeste Cawley, written by his father. It is the true story, too, of his gifted brothers and of all the Cawley family. I write also of child prodigy and genius in general: what it is, and how it is so often neglected in the modern world. As a society, we so often fail those we should most hope to see succeed: our gifted children and the gifted adults they become. Site Copyright: Valentine Cawley, 2006 +

Wednesday, May 26, 2010

Does peer review work?

Peer review is the process by which journals evaluate articles submitted to them, to decide whether to publish them. It involves the commenting on the papers by anonymous "peers" - that is, others in the field. It is generally regarded as an integral part of the process of publication and is respected by many - but is it the right way to go?

I ask for a reason. Sometimes peers are not as insightful as one might suppose. Their views may be unfounded, misconceived or simply plain wrong - yet they determine whether a particular paper is published. A recent example comes to mind. I submitted a paper to a journal and received comments in due course. What I found interesting, was the difference in approach of the reviewers. One was positive in his examination of the paper, looking at the good and suggesting ways it could be even better. He thought the paper very interesting. The other reviewer, however, was extremely aggressive and seemed personally affronted by the paper. What was very telling about his review was that he had MISUNDERSTOOD THE PAPER. His critiques were based on perceived flaws that had no relevance to the paper. He even criticized it because he wrote that something was not included. In fact, it was included and was clearly stated early on in the paper. It was clear that he had not even read the paper with any care, at all.

Guess which reviewer the editor had been more influenced by? That is right: the negative review. Yet, the negative review was plain wrong in his understanding of the paper. His criticisms, though phrased in an erudite manner, were erroneous to the core. They showed that he did not understand the experiment, or why it had been conducted in the manner it had. Yet, rather than see through the problems with this reviewers review, the editor had, it is clear, been convinced by it. The other quite positive review was ignored in its favour.

Thus it is that the reviewer who saw value in the paper was ignored and the reviewer who did not understand the paper, won out. Clearly, this is not how it should be. Reviewers determine the fate of papers, yet reviewers may not be insightful, knowledgeable, impartial or understanding. Sometimes, reviewers write responses for reasons that have nothing to do with the contents of the paper itself. For instance, the negative reviewer seemed to take exception to my co-author, from comments he made, that, perhaps, he did not realize I would see.

I think that peer reviews should not be allowed to impede the dissemination of scientific work. They should take place after publication and not before. They should be public, for all to see - and perhaps it would be better if they were not anonymous, so that certain reviewers would not be free to pursue private agendas, in secret.

All papers should be publishable at once. Then, the assessment process should begin. The system, as is, lends itself to abuse. It is too easy to suppress another's work, to frustrate another's progress, if that is one's intent. It is also rather random whether or not reviewers are good - that is, whether they understand the work and are true to it - or not.

Personally, I found it rather odd that one reviewer should have expressed himself with such forceful hostility - and yet, the other reviewer clearly appreciated the work and was of a mind to let it be published, with some well chosen improvements. It was almost as if they had been reading different papers. (Except for the fact that the hostile reviewer hadn't read it carefully at all).

There are moves, in some quarters, towards changing the way peer review is done. There are some places where peer review takes place after publication. This is the best way - from the point of view of access to knowledge. I hope to see it become the predominant paradigm, in years to come. After all, for most of the history of science, peer review just was not done at all - and that didn't stop science from advancing. Einstein's four famous papers in his "annus mirabilis", of 1905, were not peer reviewed. They were simply published because the editor of the journal, in question, thought them interesting. Perhaps, today, he would have had problems publishing them at all, facing reviewers of the kind I encountered with my own paper: ones who did not understand the work in front of them.

History shows that scientific advancement is not dependent on peer review (which only really became dominant from the middle of the last century)...so why is academia so enamoured of it? Could it be that peer review, itself, needs to be reviewed?

(If you would like to learn more of Ainan Celeste Cawley, 10, or his gifted brothers, Fintan, 6 and Tiarnan, 4, this month, please go to:
http://scientific-child-prodigy.blogspot.com/2006/10/scientific-child-prodigy-guide.html

I also write of gifted education, child prodigy, child genius, adult genius, savant, megasavant, HELP University College, the Irish, the Malays, Singapore, Malaysia, IQ, intelligence and creativity.

My Internet Movie Database listing is at: http://imdb.com/name/nm3438598/
Ainan's IMDB listing is at http://imdb.com/name/nm3305973/
Syahidah's IMDB listing is at http://imdb.com/name/nm3463926/

Our editing, proofreading and copywriting company, Genghis Can, is at http://www.genghiscan.com/

This blog is copyright Valentine Cawley. Unauthorized duplication is prohibited. Use only with permission. Thank you.)

Labels: , ,

AddThis Social Bookmark Button
posted by Valentine Cawley @ 1:13 PM 

8 Comments:

Anonymous Anonymous said...

Totally agree. My friend working as a researcher has endless stories about how "savvy" scientists are- sabotaging others work by giving it poor reviews, rejecting a paper and then copying the idea...etc

It is indeed a terrible system whereby (I'm sure) many brilliant papers have been put to rest due to the ambitions of others.

The worse thing is that there is no system to eradicate such behavior. One may complain, but the publication/journal is highly unlikely to take any action against the reviewer (as i've been told).

2:46 PM  
Blogger Valentine Cawley said...

It is sad to see how a system supposedly designed to support quality, ends up being corrupted into a means to thwart competitors and, in a way, ruin science.

Removing anonymity from the reviewers would be a great step forward: they would destroying themselves if they were unfair on others.

3:12 PM  
Blogger Fox said...

For most reputable journals, you can suggest a list of potential reviewers to the editor. With this list, you can avoid potential conflicts of interest as well as pick people who are familiar with your field of work. Obviously, if you know X is a competitor, then you won't include X in that list.

Also, if the reviewer misunderstood your paper, then it could be the case that the paper wasn't well-written. As a very famous researcher in my field once told me, "writing IS research".

8:59 PM  
Blogger Valentine Cawley said...

The journal in question is reputable, but does not allow the suggestion of reviewers. Very worryingly,also, the authors are not anonymous to the reviewers but the reviewer are anonymous to the authors. This is the worst combination from the point of view of abuses and personal agendas.

The possibility of suggesting reviewers is a poor fix to a broken system. If a person knows the field, then they are also most likely a competitor...the two factors cannot be separated. Thus the very nature of reviewing exposes ones work to the interference of competitors under the present peer review system.

If you looked into the history of peer review you should see some really shockers of peer blocking going on...some of it for research that later won the Nobel prize. It is ugly.

As for the writing: it was clearly written...but the hostile reviewer clearly had not read it carefully since some of his remarks reveal ignorance of its contents.

The other reviewer had no such issues...so the writing probably wasn't at fault.

Thanks for your perspective.

9:49 PM  
Blogger Fox said...

It is difficult for your reviewers to copy your work because the review process usually takes about 1 to 3 months. You can't do much in that 1 to 3 months. And say, even if your competitor unfairly copies your idea from your article under review, he too will also have to submit that plagiarized work to some journal where it will run the same risk of getting plagiarized.

When submitting papers to journals, a cover letter is usually required and you are allowed to write to the editors and offer them suggestions as to whom not to pick as a reviewer.

From experience, I can tell you that is definitely true of very prestigious journals such as Science, Nature, Cell, Physical Review A to E, Nano Letters, JACS, Nanotechnology, etc.

Reviewers have to be anonymous. Otherwise, there exists the opportunity for the authors to influence the reviewers. This is especially so if the authors are very influential figures in the research community. Scientists are only human after all. I've been asked to review papers by people who are very much higher up the totem pole than I and I would have hesitated to agree to be a reviewer if my identity was known to the authors.

10:27 PM  
Blogger Christine said...

I have a published copy of Einstein's papers, but I haven't gotten around to reading it yet.

12:37 PM  
Blogger Valentine Cawley said...

Hi Christine,

From your past reading, and this, it seems you have a taste for the worthy. Not many, these days, would read directly from the source, where Einstein's work was concerned.

Happy reading.

1:11 PM  
Blogger Syahidah and Valentine said...

Thank you, Fox, for your advice and perspectives, regarding publishing in journals. In future, I will see if I can avail myself of a judicious cover letter!

Sorry I didn't post your comment sooner...I have been busy.

9:13 PM  

Post a Comment

<< Home

Page copy protected against web site content infringement by Copyscape