*
One of the many flaws in peer review as applied in refereeing is that it trains scientists to become-comfortable-with untruthfulness, and to make excuses for (what they believe to be) wrong facts or ideas published under their own names.
*
This has become an almost routine aspect of the journal peer review process.
The original and proper function of a scientific paper was for an individual (or small group) to publish their views and observations under their own names, and to be responsible for these views - to defend these views against competent critique - or, if they could not defend them, eventually to retract them.
But nowadays, in order to publish, authors are frequently - probably usually - compelled to alter their arguments and interpretations in line with what journal referees suggest.
*
The modern 'scientific' paper as it appears when published in a peer reviewed journal generally ends-up as a mosaic of some of the author's own best estimates of truth (although some things the authors consider to be important will probably have been deleted at the insistence of referees), along with the various opinions and interpretations that originate from the journal referees.
In a nutshell, the modern editorial process changes what the author believes and considers important, to yield an incomplete and biased version of the authors best knowledge; but the result is published under the name of the authors, and these authors are supposed to take responsibility for it.
*
In my opinion, this is a corrupt and corrupting practice.
Except for the correction of accidental errors, articles submitted to journals ought not to be subjected to substantive changes.
The editor's role should be restricted to matters of presentation. By and large, papers should be accepted or rejected - not rebuilt.
As I used to express it when I was myself an editor; to preserve the integrity of science, the editor's role should be as a chooser, not a changer.
*
But this month I and two co-authors looked at a referee's comment which said, essentially, "Yes, but how would you measure those two constants?". So we discussed it and I wrote a para describing in detail how we could do it. It adds usefully to the merit of the paper; we are grateful to the referee for provoking us with his question.
ReplyDeleteBut I admit that I do get weary of the referees' comments that mean (I suspect) "you must cite my work more often".
That is how it is supposed to work in theory.
ReplyDeleteBut any benefits from suggested, optional, discretionary improvements triggered by referee's comments are outweighed by the harm (to outcome and to attitudes) caused by the regular imposition of compulsory referee-generated changes as a condition of publication.
Actually, the process often begins with PhD theses.
I know of situations where (for example) the student has coherently argued (with appropriate referencing) against the validity of doing analysis X, but the examiners have nonetheless insisted on the inclusion of analysis X as a condition of passing the degree.
So at his first step on the professional ladder, the newly minted doctor has been made to put his name to something he rationally believes is incorrect.
To measure the corruption you describe it is only necessary to see the degree global warming "science" is peer reviewed. We are not very familiar with EBM, but your essay might be a great example.
ReplyDeleteThe objection here is not really to peer review but to the "revise and resubmit" option and to its necessary concomitant, the hideous "comments for the authors." R&R is, on balance, a bad thing. It essentially recruits referees as uncredited co-authors, to the detriment of referees, authors, and science.
ReplyDeleteWhen I write a referee report, I feel that I have slacked severely if it is less than one single spaced typewritten page and slacked somewhat if it is less than two. Evaluating a paper can be done in two short paragraphs---I know because the middle two paragraphs in my letter to the editor always contain such an evaluation.
It's hard, time-consuming (unpaid!) work to write two or more pages of useful comments on a topic you are only moderately interested in. It makes you inflexible when authors resist. And it makes editors weak---if you make a practice of blowing off your referees, they will return the favor.
The journal Economic Inquiry put in place an optional "no R&R" submission track some years ago. Other economic journals have not followed suit for whatever reason.
@Bill - this particular objection to peer review was to the corrupting effects of R&R - but I have many other objections in addition!
ReplyDeletePeople can always create an idealized view of how peer review *might* be - but its actual effect has everywhere been to destroy real science and assimilate it to generic bureaucracy.