My 1997 knock-down, drag-out review of Evidence-based medicine: how to practice and teach EBM; by Sackett DL, Richardson WS, Rosenberg W, Haynes RB

Charlton BG. Book Review: Evidence-based medicine: how to practice and teach EBM by Sackett DL, Richardson WS, Rosenberg W, Haynes RB. [Churchill Livingstone, Edinburgh, 1997]. Journal of Evaluation in Clinical Practice. 1997; 3: 169-172

The truth is out! EBM is not just a fancy new name for Clinical Epidemiology, nor a novel approach to continuing education, nor simply the latest excuse for managers and politicians to take-over clinical practice. Evidence-based medicine is a fundamentalist evangelical cult: there is no God but EBM: Sackett is its prophet and this little book its bible.

Evidence-based MEDICINE is an astonishing volume (not least in the title: written as Evidence-based MEDICINE - with medicine capitalized, as if this book were actually about ‘medicine’, the whole thing - or everything that mattered, anyway). Never in the history of publishing has so much irritation been concentrated into so small a package. EBM is compact in the same way that the thoughts of Chairman Mao is compact, and for the same reason - we are intended to carry it with us always and refer to it often. And, just like my teenage copy of the Little Red Book, EBM has a eazee-wype vinyl cover so that it will be durable and resistant to spillages and secretions, coffee cups and coughed-up sputum - even at the front line of clinical activity. Clearly this is a book that is not meant to be kept back at the ward office, but must be with the doctor at all times, nestling on the hip, a comforting presence.

I don’t know whether the advocates of EBM have noticed the lukewarm reception their creed has been accorded by practicing clinicians and clinical scientists, and the fact that most of their supporters (and the most zealous supporters) come from epidemiology, biostatistics, health economics and management. If they have noticed this, their response has been to assume that it was due to a misunderstanding about the nature of EBM, and that if only they explain what EBM is about (in louder and louder voices) eventually the laggards will come to share their enthusiasm.

It doesn’t seem to be happening. Critics continue to believe that the techniques of EBM are flawed, its conceptualization of medical practice radically deficient, its remedies narrow and simplistic. And the incomprehension comes, not from the ‘traditional’ clinicians but from the EBM advocates who - for all their statistical sophistication - seem to be incapable of understanding the limitations of their approach. I mean this literally. Neither in this book, nor in lectures by its advocates, nor anywhere in the literature of EBM of which I am aware, have I ever detected any glimmering of comprehension as to the fundamental and damaging force of the criticisms that have been leveled against EBM.

None of this would matter much if EBM were more modest in its claims. But the new name gives the game away. Evidence based medicine used to be called Clinical Epidmiology, which was a good name, describing the major methodological doctrine of the activity - the application of epidemiological information to clinical practice. This activity is, I would argue, more difficult and problematic than the authors suggested, but Clinical Epidemiology is a perfectly reasonable and legitimate branch of medical practice. But renaming the activity Evidence-based medicine strikes one as a rhetorical move that arrogantly (and without justification) begs all the important questions about medical practice. If picking-out the good bits from Clinical Epidemiology is disbarred, and the choice presented is the stark one of all of EBM or nothing, then EBM must be rejected.

The EBM bandwagon is getting rather beyond a joke. This book exudes a blithe spirit, everything in the garden is rosy. Yet, time and again it has been pointed-out that the techniques advocated by EBM cannot do what they claim to do. For instance, skepticism concerning meta-analysis is widespread and deep-rooted among clinicians and medical scientists. The fundamental critique of this technique is ignored completely, and instead the authors busy themselves with raising and answering more superficial problems and quibbles, often of a statistical nature, apparently in order to create the impression of rigor and to try and convince the reader that every possible objection has been answered.

This cannot go on indefinitely. Eventually the advocates of EBM will have to descend from their citadels and debate with their critics.

The book opens with a Preface that is a masterpiece of smugness and self-mythologizing. There are potted biographies of all the writers (a self-styled ‘group of rebels’), with tales of their conversion-experiences. Their credentials are listed: military bravery, unstinting dedication to patients, relentless self-education, skepticism (wonder what happened to that on the way to this volume?). And as we embark on the main text, the authors leave us with the four commandments necessary for EBM to be effective: ‘master’ clinical skills; practice continuous life-long, self-directed learning; be humble (I kid you not, they really do say this) and finally be enthusiastic, irreverent and fun-loving. And that is all you need to make EBM work - X-ray vision and bulletproof underpants are merely optional extras.

The book is dedicated to Kilgore Trout. This gives the game away - Trout is the fictional alter ego of the science fiction author Kurt Vonnegut - doyen of the sixties counter culture. So EBM - authored principally by a Professor at Oxford and McMaster Universities and one of the most famous doctors in the world - is dedicated to an invented, obscure, deluded scribe of pulp Sci Fi. This is telling. In a sense EBM is a pompous artifact into which the medical version of anti-establishment radicalism has declined during the nineteen eighties, in another sense it is the sad monument to the bad faith of those who sacrificed their idealism on the altar of worldly power and success.

Fair enough and good luck to them; but you can’t have it both ways guys. You might have been wild boys in your youth, but now you are the establishment: darlings of Health Service Management; funded to the hilt by the Department of Health; an orthodoxy enforced by contract and regulations. Whatever luster of radical chic EBM once had has been long tarnished as the spirit of reform was melted down and recast into a model of entrepreneurial self-promotion. Skeptical reflection was re-written by systematization and conversion into algorithmic and statistical routine.

This is a book of wild-eyed zeal - never have I read anything to compare with it in medicine: this makes the polemic of Ivan Illich look like a model of Mandarin impartiality. What EBM most resembles are those self-help books that you can buy in airports, telling you how to win friends, influence people, manage multinational corporations, and make a million bucks.

Of course there are good ideas and reasonable points amongst the flight of ideas - after all, the original 1985 Clinical Epidemiology volume from which EBM descended, was an innovative, fascinating and useful book. But nowadays the gloves are off, EBM is not an element of good practice, it is good practice. EBM has moved from fringe cult to world religion, with a systematic theology to rival Thomas Aquinas.

Evidence-based MEDICINE describes and advocates an interlocking system of training, canonical textbooks, commentaries, mantras, technical fetishes, icons, statistical high priests and charismatic preaching. The acolyte is to be sustained in his or her lifelong devotions by regular confession (in EBM journal clubs), a web page, a journal or two, and a local spiritual centre (Oxford - where else?). As yet, there is no requirement to bow to McMaster in the west after ward rounds - but this cannot be long in coming, and pilgrimages to the source have been on the go for many years.

The EBM cult has that hermetically sealed, circular irrefutability (within its own terms) that characterizes Marxism, psychoanalysis and postmodernism and the other ‘isms’ that have defaced Western civilization. Like the Mooney cult, or Scientology, EBM makes its appeal to the young, the uneasy, the unconfident. The first step is to arouse unease, then to offer certainty and security within the fold. The clinical mistakes of the past are paraded. A few carefully chosen examples are set forth where it is argued that if only EBM techniques such as meta-analysis had been around, and if only the wise commissars of management had been empowered to enforce the findings upon a recalcitrant profession, then certain large (and quantified) benefits would have accrued. The eye of narrow and retrospective wisdom is them swept around to a broad and uncertain future, and it is assumed that EBM techniques offer a sure and certain short-cut to knowledge across the board of practice.

The key is to be information. A defining feature of EBM is that it assumes that the answers to clinical questions lie ‘latent’ within existing data, or at worst within data from studies such a megatrials, the principles of which have been reduced to a routine, and the only barriers to which are lack of funding (and those stubborn doctors). EBM works by obtaining an overview of the whole data set, and by interrogating this data set using various epidemiological and biostatistical techniques which can be summarized in checklist form.

So that EBM is a set of techniques, not a practice. It is an arbitrary technique, in that its validity is based only upon highly selective, post hoc reasoning, swift and generalized appeals to ‘common sense’, and especially the mind-numbing effect of large data bases and complex statistical analyses which affect clinicians like the headlights of an oncoming car affect a frightened rabbit.

Yet the reality is that EBM is a vast edifice built upon foundations of sand. It is an incomplete and grossly over-simplified approach to medicine, yet is complex enough that that its partial and simplistic nature is disguised from the causal observer. Mastery of its formulae is easy enough to be attainable by anyone keen enough; but hard enough to mean that some degree of training is required, and some degree of status will accrue from this rarity value.

The basic error of EBM is quite simple. It is that epidemiological data does not provide the information necessary to treat individual patients. This error is intractable and intrinsic to the nature of the methodological nature of epidemiology, and no amount of statistical jiggery-pokery with huge amounts of data can make any difference to it. The edifice of EBM is like a huge attempt to distract out attention from this flaw. But - as the great physicist Richard Feynman said in the relation to the Challenger disaster - nature will not be fooled.

In the end the house of cards erected by EBM will come tumbling down because it cannot deliver what it promises. It offers statistical pseudo-precision, but clinicians will rapidly become aware that precision of predictions derived from mega-randomized trials, or meta-analyses, bears no necessary relationship at all to the precision of prediction of what will happen in their individual patients.

So, although the hubris of EBM is both dangerous and damaging, we can at least be confident that it will not last*. In the meantime we will have to tolerate the waste of time and resources which should be spent on more wholesome or useful things, the erection of new barriers of red tape and regulation, and the construction of a new set scientifically-bogus but rhetorically-effective tools of managerial authority. But hardest to tolerate will be watching the rise and rise of a gang of posturing tenured-radicals whose ascent of the greasy pole of status will be interrupted only by pausing to contemplate their own extraordinary irreverence, enthusiasm and (pass the sick bag Alice…) humility.


*Note: On the contrary, it did last - see my paper The Zombie Science of EBM from 12 years later.