2/15/2010

What does "peer-review" for scientific and technical publications actually mean?

Not as much as you might think!

Chicago Boyz: Scientific Peer-Review is a Lightweight Process

Oh, and by the way, I, 1389, formerly did some anonymous tech reviewing of IT- and computing-related manuscripts for a reputable publisher. Even though it paid very little, I did the best job I possibly could. My task was simply to make note of any deficiencies, errors, or omissions in the manuscript. Problems with the manuscript might include anything from unclear writing and garbled logic to poor methodology, obsolete information, and blatant factual errors.

That said, I was not tasked with fact-checking every assertion each author made. Nor was I expected to set up a test system to reproduce all the results from each author's computer program examples.

By and large, this peer review process is a good thing for publishers, in that it helps to weed out unsaleable manuscripts that are overloaded with trite observations, emotion-driven personal agendas, or incomprehensible drivel. That said, the peer review process is not capable of proving that the assertions in any manuscript are "right" or "wrong." Nor is the peer review process capable of withstanding an organized propaganda campaign bent on using scientific or technical expertise to bolster a partisan point of view.

The long and the short of it is that one cannot assume that the contents of a publication have been proved "correct" or represent a "scientific consensus" merely because the publication has been "peer-reviewed."

No comments: