Comments on the Peer-Review Process

Over and over, the avid proponents of catastrophic anthropogenic global warming (AGW) protest that I must be wrong in challenging the bad science backing this hypothesis, or theory in their terms.  After all, the preponderance of published papers in the refereed climate journals back the idea of catastrophic AGW and I must honor the peer-review process, they argue.  Yes, they are right that the majority of published journal papers back catastrophic AGW or at least the reasonable possibility of catastrophic AGW.

But, being a scientist, I have read many published scientific papers in fields much less politicized than that of climatology and I can tell you that there are many, many papers representing flat out bad science or mixtures of good and bad science which are published in peer-reviewed journals.  In addition, I have refereed many papers submitted for publication and I have often been the only reviewer who raised serious issues against publication or who fought hard for changes as a condition of publication.  Frankly, in our non-confrontational age, few want to confront paper authors with their slip-ups and any incompetency that may be revealed in a paper.  We live in a society that really wants to give an A to every student.

What is more, the reviewer must often work very hard to be sure that he has valid points of contention.  The scientists submitting the work probably spent at least a few months doing the research they are trying to describe and then spent a few weeks writing the article.  No reviewer is likely to have so much time to devote to the same work, yet he is tasked with confirming it as valid or declaring it invalid or at least unproven.  Being a good reviewer takes a lot of time and the rewards are very few.

Oftentimes, my objections have taken the form of this or that conclusion is unproven for this or that reason.  But, surprisingly, I have also rejected many submissions on the grounds that data actually in the paper proves that some of the conclusions are wrong.  It is amazing to me how often authors trip themselves up by either using faulty logic or even more often by having blinders on with respect to the conclusions they believe to be the right ones.  In other words, even in non-politicized science, the authors often are able to ignore their own data and results which are in conflict with the conclusions they apparently wanted to reach.  Frankly, in some cases, I have even wondered if they were not de-emphasizing results that were inconvenient in that they muddied the waters with respect to the conclusions they wanted to reach.  This is dishonest.  It is bad science.  And, it is not unusual.

For young academic scientists, the pressure to publish is huge.  It leads to hastily prepared papers and an attempt to publish as many papers as possible, even if the conclusions of the paper do little, if anything, to advance our knowledge of science.  The reviewers, while nominally anonymous, nonetheless will in many cases be revealed by the nature of their objections or the knowledge they reveal in those objections.  If the journal is associated with a particular professional scientific society, the number of fellow members you might anger over time may be appreciable.  If you hope to become an elected officer of the society or a fellow of the society, it is not politic to take your duties as a reviewer very seriously.  It is too risky.  It is better to go along and get along.  It is career-advancing to allow bad science to be published and then to count on critical readers to sort out the chaff from the wheat.

Frankly, I am not very trusting that a paper represents valid science simply since it was published.  I read the paper and try to conclude from the argument made in the paper whether the science is good or not.  Sometimes, I conclude that it is bad, sometimes that it is probably good, and sometimes that the situation is indeterminate.  This indeterminate case may occur because while the author's conclusions appear consistent with the data and results presented and while those results may be useful in advancing the evidence for a theory, there is still much work that needs to be done before the theory is proven right.  One might often say this represents good science because this is part of the proper process.  However, such work may often be bad science in which the authors have selected only that part of their data and results which are consistent with the theory they are advancing as correct.  There are many cases in which negative results are simply left out.  This may be done because the author thinks the paper is more important if it purports to prove or advance a particular theory, or it may be because the author has already staked a substantial part of his reputation on a particular theory.  Some scientists simply cannot admit to having been wrong.