The Zombie science of Evidence-Based Medicine (EBM): a personal retrospective
Bruce G Charlton. Journal of Evaluation in Clinical Practice. 2009; 15: 930-934.
Professor of Theoretical Medicine
University of Buckingham
e-mail: bruce.charlton@buckingham.ac.uk
***
Abstract
As one of the fairly-frequently cited critics of the socio-political phenomenon which styles itself Evidence-Based Medicine (EBM), it might be assumed that I would be quite satisfied by having collected some scraps of fame, or at least notoriety, from the connection. But in fact I now feel a fool for having been drawn into criticizing EBM with the confident expectation of being able to kill it before it had the chance to do too much harm. A 'fool' not because my criticisms of EBM were wrong – EBM really is just as un-informed, confused and dishonest as I claimed it was. But because, with the benefit of hindsight, it is obvious that EBM was from its very inception a Zombie science – a lumbering hulk reanimated from the corpse of Clinical Epidemiology. And a Zombie science cannot be killed because it is already dead. A Zombie science does not perform any scientific function, so it is invulnerable to scientific critique since it is sustained purely by the continuous pumping of funds. The true function of Zombie science is to satisfy the (non-scientific) needs of its funders – and indeed the massive success of EBM is that it has rationalized the takeover of UK clinical medicine by politicians and managers. So I was simply wasting my time by engaging in critical evaluation of EBM using the normal scientific modes of reason, knowledge and facts. It was useless my arguing against EBM because a Zombie science cannot be stopped by any method short of cutting-off its fund supply.
***
It is pointless trying to kill the un-dead
Since I am one of the fairly-frequently cited critics of the socio-political phenomenon which styles itself Evidence-Based Medicine (EBM), it might be assumed that I would be quite satisfied by having collected some scraps of fame, or at least notoriety, from the connection [1-4]. But in fact I feel rather a fool for having been drawn into criticizing EBM. Not because my criticisms were wrong – of course they weren’t. EBM really is just as uninformed, confused and dishonest as I claimed in my writings. But because - with the benefit of hindsight - it is obvious that EBM was, from its very inception, a Zombie science: reanimated from the corpse of Clinical Epidemiology.
I recently delineated the concept of Zombie science [5], partly based on my experiences with EBM, and the concept has already spread quite widely via the internet. Zombie science is defined as a science that is dead but will not lie down. Instead, it keeps twitching and lumbering around so that it somewhat resembles Real science. But on closer examination, the Zombie has no life of its own (i.e. Zombie science is not driven by the scientific search for truth [6]); it is animated and moved only by the incessant pumping of funds. Funding is the necessary and sufficient reason for the existence of Zombie science; which is kept moving for so long as it serves the purposes of its funders (and no longer).
So it is a waste of time arguing against Zombie Science because it cannot be stopped by any method except by cutting-off its fund supply. Zombie science cannot be killed because it is already dead.
When dead fish seem to swim
I ought to have seen all this more quickly, because I knew Clinical Epidemiology (CE) [7] before it was murdered and reanimated as an anti-CE Zombie, and I was by chance actually present at more-or-less the exact public moment when – by David Sackett, in Oxford, in 1994 - the corpse of CE was galvanized into motion in the UK, and began to be inflated by rapid infusion of NHS funding on a massive scale. The process was swiftly boosted by zealous promotion from the British Medical Journal, which turned-itself into a de facto journal of EBM then went on to benefit from the numerous publishing and conference opportunities created by their advocacy [3].
I ask myself now; how I could have been so naïve as to imagine that EBM – born of ignorance and deception - would be open to the usual scientific processes of conjecture and refutation? From the very beginning, from the very choice of the name 'evidence-based medicine'; it was surely obvious enough to the un-blinkered gaze that we were dealing here with something outwith the kin of science. Why couldn’t I see this? Why was I so ludicrously reasonable?
The fact is that I was lured into engagement by pride: pride at seeing-through the clouds of smoke which were being deployed to obscure the origins of EBM in CE; pride at recognizing the numerous ‘moves’ by which unfounded assertion was being disguised as evidential - at the playing fast-and-loose with definitions of ‘evidence’ in order to reach pre-determined conclusions… I think that I was excited by the possibilities of engaging in what looked like a easy demolition job. My only misgiving was that it was too easy – destroying the scientific basis of EBM would be about as challenging as shooting fish in a barrel.
Well, it was as easy as that. But shooting fish in a barrel is a pointless activity if nobody is interested in the vitality of the fish. As it turned-out – the edifice of EBM could be supported as easily by a barrel of dead fish as by one full of lively swimmers. So long as the fish corpses were swirling around (stirred by the influx of research funding) then, for all anyone could see at a quick glance (which is the only kind of glance most people will give), it looked near-enough as if the fish were still alive.
Anyway, undaunted, I and many others associated with the Journal of Evaluation in Clinical Practice set about the work of analyzing, selecting and discarding among the assertions and propositions of EBM.
There was the foundational assertion that in the past pre-EBM medicine had not been based on evidence but on a blend of prejudice, tradition and subjective whim; this now to be swept aside by the ‘systematic’ use of ‘best evidence’. This was an ignorant and unfounded belief – coming as it did after the (pretty much) epidemiology-unassisted 'golden age' of medical therapeutic discovery peaking somewhere between about 1940 and 1970 [8-11].
With regard to ‘best evidence’ there was the assertion that ‘evidence’ meant only focusing on epidemiological data (and not biochemistry, genetics, physiology, pharmacology, engineering or any other of the domains which had generated scientific breakthroughs in the past). It meant ignoring the role of ‘tacit knowledge’ derived from apprenticeship. And it was clearly untrue [12-15].
Then there was the assertion that the averaged outcomes of epidemiological data, specifically randomized trails and their aggregation by meta-analysis of RCTs were straightforwardly applicable to individual patients. This was a mistake [12, 16-18].
On top of this there was the methodological assertion that among RCTs the ‘best’ were the biggest – the ‘mega-trials’ which attempted to maximize recruitment and retention of subjects by simplifying methodologies and thereby reducing the level of control. This was erroneous [12, 16, 19].
In killing-off the bottom-up ideals of Clinical Epidemiology, EBM embraced a top-down and coercive power structure to impose EBM-defined ‘best evidence’ on clinical practice [20, 21]; this to happen whether clinical scientists or doctors agreed that the evidence was best or not (and because doctors have been foundationally branded as prejudiced, conservative and irrational –EBM advocates were pre-disposed to ignore their views anyway).
Expertise was arbitrarily redefined in epidemiological and biostatistical terms, and virtue redefined as submission to EBM recommendations – so that the job of physician was at a stroke transformed into one based upon absolute obedience to the instructions of EBM-utilizing managers [3].
(Indeed, since too many UK doctors were found to be disobedient to their managers; in the NHS this has led to a progressive long-term strategy of the replacing doctors by more-controllable nurses, who are now first contact for patients in many primary and specialist health service situations.)
Biting-off the hand that offers EBM
The biggest mistake made in analyzing the EBM phenomenon is to assume that the success of EBM depended upon the validity of its scientific or medical credentials [22]. This would indeed be reasonable if EBM were a Real science. But EBM was not a Real science, indeed it wasn’t any kind of science at all as was clearer when it had been correctly characterized as a branch of epidemiology, which is a methodological approach sometimes used by science [13-15].
EBM did not need to be a science or a scientific methodology, because it was not adopted by scientists but by politicians, government officials, managers and biostatisticians [3]. All it needed – scientifically – was to look enough like a scientific activity to convince a group of uninformed people who stood to benefit personally from its adoption.
So, the sequence of falsehoods, errors, platitudes and outrageous ex cathedra statements which constituted the ideological foundation of EBM, cannot be – and is not - an adequate or even partial explanation for the truly phenomenal expansion of EBM. Whether EBM was self-consciously crafted to promote the interests of government and management, or whether this confluence of EBM theory and government need was merely fortuitous, is something I do not know. But the fact is that the EBM advocates were shoving at an open door.
When the UK government finally understood that what was being proposed was a perfect rationale for re-moulding medicine into exactly the shape they had always wanted it - the NHS hierarchy were falling over each other in their haste to establish this new orthodoxy in management, medical education and in founding new government institutions such as NICE (originally meaning the National Institute for Clinical Excellence – since renamed [20]).
As soon as the EBM advocates knocked politely to offer a try-out of their newly-created Zombie; the door was flung open and the EBM-ers were dragged inside, showered with gold and (with the help of the like-minded Cochrane Collaboration and BMJ publications) the Zombie was cloned and its replicas installed in positions of power and influence.
Suddenly the Zombie science of EBM was everywhere in the UK because money-to-do-EBM was everywhere – and modern medical researchers are rapidly-evolving organisms which can mutate to colonize any richly-resourced niche – unhampered by inconveniences such as truthfulness or integrity [23]. Anyway, when existing personnel were unwilling, there was plenty of money to appoint new ones to new jobs.
The slaying of Clinical Epidemiology (CE)
But how was the Zombie created in the first place?
In the beginning, there had been a useful methodological approach called Clinical Epidemiology (CE), which was essentially the brainchild of the late Alvan Feinstein – a ferociously intelligent, creative and productive clinical scientist who became the senior Professor of Medicine at Yale and recipient of the prestigious Gairdner Award (a kind of mini-Nobel prize). Feinstein's approach was to focus on using biostatistical evidence to support clinical decision making, and to develop forms of measurement which would be tailored for use in the clinical situation. He published a big and expensive book called Clinical Epidemiology in 1985 [24]. Things were developing nicely.
The baton of CE was then taken up at McMaster University by David Sackett, who invited Feinstein to come as a visiting professor. Sackett turned out to be a disciple easily as productive as Feinstein; but, because he saw things more simply than Feinstein, Sackett had the advantage of a more easily understood world-view, prose style and teaching persona. So when Sackett and co-authors also published a book entitled Clinical Epidemiology in 1985 [7] –Sackett's book was less complex, less massive and much less expensive. And Sackett swiftly became the public face of Clinical Epidemiology.
But in this 1985 book, Sackett cited as definitive his much earlier 1969 definition of Clinical Epidemiology, which ran as follows: “I define clinical epidemiology as the application, by a physician who provides direct patient care, of epidemiologic and biometric methods to the study of diagnostic and therapeutic process in order to effect an improvement in health. I do not believe that clinical epidemiology constitutes a distinct or isolated discipline but, rather, that it reflects an orientation arising from both clinical medicine and epidemiology. A clinical epidemiologist is, therefore, an individual with extensive training and experience in clinical medicine who, after receiving appropriate training in epidemiology and biostatistics, continues to provide direct patient care in his subsequent career” [25] (Italics are in the original.).
Just savour those words: ‘by a physician who provides direct patient care’ and ‘I do not believe that clinical epidemiology constitutes as distinct or isolated discipline… but, rather, an orientation’. These primary and foundational definitions of clinical epidemiology were to be reversed when the subject was killed and reanimated as EBM which was marketed as a ‘distinct and isolated discipline’ (with its own training and certification, its own conferences, journals and specialized jobs) that was being practiced many individuals (politicians, bureaucrats, managers, bio-statisticians, public health employees…) who certainly lacked ‘extensive’ (or indeed any) ‘training and experience in clinical medicine’; and who certainly did not provide direct patient care.
I came across Sackett's Clinical Epidemiology book in 1989 and was impressed. Although I recognized that CE ought to be considerably less algorithm-like and more judgment-based than the authors suggested even in 1985; nonetheless I recognized that Clinical Epidemiology was a fresh, reasonable and perfectly legitimate branch of knowledge with relevance to medical practice. And Clinical Epidemiology was a good name for the new subject, since it described the methodological nature of the activity – which was concerned with the importance of epidemiological methods and information to clinical practice.
But during the period from 1990-92, Clinical Epidemiology was first quietly killed then loudly reanimated as Evidence-Based Medicine [22]. In retrospect we can now see that this was not simply the replacement of an honest name with a dishonest one that arrogantly and without justification begs all the important questions about medical practice. Nor was it merely the replacement of the bottom up model of Clinical Epidemiology with the authoritarian dictatorship which EBM rapidly became.
No, EBM was much more radically different from Clinical Epidemiology than merely a change of name and an inversion of authority; because the new EBM sprang from the womb fully-formed as a self-evident truth [3, 4]. EBM was not a hypothesis but a circular and self-justifying revelation in which definition supported analysis which supported definition – all rolled-up in an urgent moral imperative. To know EBM was to love him; and to recognize him as the Messiah; and to anticipate his imminent coming.
Therefore EBM was immune to the usual feedback and critique mechanisms of science; EBM was not merely disproof-proof but was actually virtuous – and failure to acknowledge the virtuous authority of EBM and adopt it immediately was not just stupid but wicked!
(This moralizing zeal was greatly boosted by association with the Cochrane Collaboration including its ubiquitous spiritual leader, Sir Iain Chalmers.)
In short, EBM was never required to prove itself superior to the existing model of medical practice; rather, existing practice was put into the position of having to prove itself superior to the newcomer EBM!
Zombies with translucent skins
Just think of it, for a moment. Here was a doctrine which advocated rejecting and replacing-with-itself the whole mode of medical science and practice of the past. It advocated a new model of health service provision, new principles for research funding, a new basis for medical education. And the evidence for this? Well… none. Not one particle. ‘Evidence-based’ medicine was based on zero evidence.
As Goodman articulated (in perhaps the best single sentence ever written on the subject of EBM) “…There is no evidence (and unlikely ever to be) that evidence-based medicine provides better medical care in total than whatever we like to call whatever went before…” [26]. So EBM was never required to prove with evidence what it should have been necessary to prove before beginning the wholesale reorganization of medical practice: i.e. that EBM was a better system than ‘whatever we like to call’ whatever went before EBM.
Had anyone done any kind of side-by-side prospective comparison of these two systems of practicing medicine before rejecting one and adopting the other? No. They could have don it, but they didn’t. The message was that EBM was just plain better: end of story.
But how could this happen? – why was it that the medical world did not merely laugh in the metaphorical face of this pretender to the crown? The answer was money, of course; because EBM was proclaimed Messiah with the backing of serious amounts of UK state funding. Indeed, it is now apparent that the funding was the whole thing. If EBM was a body, then the intellectual content of EBM is merely a thin skin of superficial plausibility which covers innards that consist of nothing more than liquid cash, sloshing-around.
Indeed, the thin skin of the EBM Zombie was a secret to its success. The EBM zombie has such a thin skin of plausibility that it is transparent, and observers can actually see the money circulating beneath it. The plausibility was miraculously thin! This meant that EBM-type plausibility was democratically available to everyone: to the ignorant and to the unintelligent as well as the informed and the expert. How marvelously empowering! What a radical poke in the eye for the arrogant ‘establishment’! (And the EBM founders are all outspoken advocates of the tenured radicalism of the ‘60s student generation [4].)
Compared with learning a Real science, it was facile to learn the few threads of EBM jargon required to stitch-together your own Zombie skin using bits and pieces of your own expertise (however limited); then along would come the UK government and pump this diaphanous membrane full of cash to create a fairly-realistic Zombie of pseudo-science. In a world where scientific identity can be self-defined, and scientific status is a matter of grant income [11], then the resulting inflatable monster bears a sufficient resemblance to Real science to perform the necessary functions such as securing jobs or promotions, enhancing salary and status.
The fact that EBM was based upon pure and untested assertions therefore did not weaken it in the slightest; rather the scientific weakness was itself a source of political strength. Because, in a situation where belief in EBM was so heavily subsidized, it was up to critics conclusively to prove the negative: that EBM could not work. And even when conclusive proof was forthcoming, it could easily be ignored. After all, who cares about the views of a bunch of losers who can’t recognize a major funding opportunity when they see it?
Content eluted, only power remains
Things got even worse for those of us who were pathetically trying to stop a government-fuelled Zombie army using only the peashooters of rational debate and the catapults of ridicule. Early EBM made propositions which were evidently wrong, but their recommendations did at least have genuine content. If you installed the EBM clockwork and turned the handle; then what came out was predictable and had content. EBM might have told you wrong things to do; at least it told you what to do with words that had meaning.
But then there was the stunning 1996 U-turn in the BMJ (where else?), in which the advocates of EBM suddenly announced a U-turn, they de-programmed their Zombies. “Evidence based medicine is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients” [27].
(Pause to allow the enormity of this statement to sink in…)
At a stroke the official meaning of EBM was completely changed, a vacuous platitude was substituted, and henceforth any substantive methodological criticism was met by a rolling-out and mantra-like repetition of this vacuous platitude [28].
Recall, if you will, Sackett’s foundational definition of CE as done: “by a physician who provides direct patient care’ and ‘I do not believe that clinical epidemiology constitutes as distinct or isolated discipline… but, rather, an orientation’. To suggest that EBM represented a ‘sell-out’ of clinical epidemiology is seriously to understate the matter: by 1996 EBM was nothing short of a total reversal of the underlying principles of CE. Alvan Feinstein, true founder of (real) Clinical Epidemiology, considered EBM intellectually laughable – albeit fraught with hazard if taken seriously [e.g. 29].
This fact renders laughable such assurances as: “Clinicians who fear top down cookbooks will find the advocates of evidence based medicine joining them at the barricades.” Wow! Fighting top-down misuse of EBM at the ‘barricades’, no less…
Satire fails in the face of such thick-skinned self-aggrandizement.
Instead of being based only on epidemiology, only on mega-RCTs and their ‘meta-analysis’, only on simple, explicit and pre-determined algorithms - suddenly all kinds of evidence and expertise and preferences were to be allowed, nay encouraged; and were to be ‘integrated’ with the old fashioned RCT-based stuff. In other words, it was back to medicine as usual – but this time medicine would be controlled from above by the bosses who had been installed by EBM.
And having done something similar with Clinical Epidemiology, and now operating in the ‘through-the-looking-glass’ world of the NHS, of course they got away with it! Nobody batted an eyelid. After all, reversing definitions while retaining an identical technical terminology and an identical organizational structure is merely politics as usual. "When I use a word," Humpty Dumpty said in a rather a scornful tone, "it means just what I choose it to mean – neither more nor less."
However much its content was removed or transformed, they still continued calling it EBM. By the time an official textbook of EBM appeared [28], clinical epidemiology had been airbrushed from collective memory, and Sackett’s 1969 clinical-epidemiologist self been declared an ‘unperson’.
Nowadays EBM means whatever the political and managerial hierarchy of the health service want it to mean for the purpose in hand. Mega-randomized trails are treated as the only valid evidence until this is challenged or the results are unwelcome, when other forms of evidence are introduced on an ad hoc basis. Clinical Epidemiology is buried and forgotten.
But a measure of success is that the NHS hierarchy who use the EBM terminology are the ones with power to decide its official meaning when deployed on each specific occasion. The ‘barricades’ have been stormed. The Zombies have taken over!
References
1. Charlton BG. Restoring the balance: Evidence-based medicine put in its place. Journal of Evaluation in Clinical Practice, 1997; 3: 87-98.
2. Charlton BG. Review of Evidence-based medicine: how to practice and teach EBM by Sackett DL, Richardson WS, Rosenberg W, Haynes RB. [Churchill Livingstone, Edinburgh, 1997]. Journal of Evaluation in Clinical Practice. 1997; 3: 169-172
3. Charlton BG, Miles A. The rise and fall of EBM. QJM, 1998; 91: 371-374.
4. Charlton BG. Clinical research methods for the new millennium. Journal of Evaluation in Clinical Practice 1999; 5: 251-263.
5. Charlton BG. Zombie science: A sinister consequence of evaluating scientific theories purely on the basis of enlightened self-interest. Medical Hypotheses, Volume 71, Issue 3, Pages 327-329.
6. Charlton BG. The vital role of transcendental truth in science. Medical Hypotheses. 2009; 72: 373-6.
7. Sackett DL, Haynes RB, Tugwell P. Clinical Epidemiology: a basic science for clinical medicine. Boston: Little, Brown, 1985.
8. Horrobin, D.F. 1987 Scientific medicine – success or failure?. In: Oxford Textbook of Medicine. 2nd ednD.J. Weatherall, J.G.G. Ledingham & D.A. Warrell) Oxford University Press, Oxford.
9. Wurtman, R.J. & Bettiker, R.L. 1995 The slowing of treatment discovery. 1965–95. Nature Medicine, 1, 1122 1125.
10. Le Fanu J. The rise and fall of modern medicine. Little, Brown: London, 1999.
11. Charlton BG, Andras P. Medical research funding may have over-expanded and be due for collapse. QJM. 2005; 98: 53-5.
12. Charlton BG. The future of clinical research: from megatrials towards methodological rigour and representative sampling. Journal of Evaluation in Clinical Practice, 1996; 2: 159-169.
13. Charlton BG. Should epidemiologists be pragmatists, biostatisticians or clinical scientists? Epidemiology, 1996; 7: 552-554.
14. Charlton BG. The scope and nature of epidemiology. Journal of Clinical Epidemiology, 1996; 49: 623-626.
15. Charlton BG. Epidemiology as a toolkit for clinical scientists. Epidemiology, 1997; 8: 461-3
16. Charlton BG. Mega-trials: methodological issues and implications for clinical effectiveness. Journal of the Royal College of Physicians of London, 1995; 29: 96-100.
17. Charlton BG. The uses and abuses of meta-analysis. Family Practice, 1996; 13: 397-401.
18. Charlton BG, Taylor PRA, Proctor SJ. The PACE (population-adjusted clinical epidemiology) strategy: a new approach to multi-centred clinical research. QJM, 1997; 90: 147-151.
19. Charlton BG. Fundamental deficiencies in the megatrial methodology. Current Controlled Trials in Cardiovascular Medicine. 2001; 2: 2-7.
20. Charlton BG. The new management of scientific knowledge in medicine: a change of direction with profound implications. In A Miles, JR Hampton, B Hurwitz (Eds). NICE, CHI and the NHS reforms: enabling excellence or imposing control? Aesculapius Medical Press: London, 2000. Pp. 13-31.
21. Charlton BG. Clinical governance: a quality assurance audit system for regulating clinical practice. (Book Chapter). In A Miles, AP Hill, B Hurwitz (Eds) Clinical governance and the NHS reforms: enabling excellence of imposing control? Aesculapius Medical Press: London, 2001. Pp. 73-86.
22. Daly J. Evidence-based medicine and the search for a science of clinical care. Berkeley & Los Angeles: Univesrity of California Press, 2005.
23. Charlton BG. Are you an honest scientist? Truthfulness in science should be an iron law, not a vague aspiration. Medical Hypotheses. Doi:10.1016/j.mehy.2009.05.009, in the press.
24. Feinstein AR. Clinical Epidemiology: the architecture of clinical research. Philidelphia: WB Saunders, 1985.
25. Sackett DL. Clinical Epidemiology. American Journal of Epidemiology. 1969; 89: 125-8.
26. Goodman NW. Anaesthesia and evidence-based medicine. Anaesthesia. 1998; 53: 353-68.
27. Sackett DL, Rosenberg WMC, Muir Gray JA, Haynes RB, Richardson WS. Evidence-based medicine: what it is and what it isn’t. BMJ 1996; 312: 71-2.
28. Sackett DL. Richardson WS, Rosenberg W, Haynes RB. Evidence-based medicine: how to practice and teach EBM. Edinburgh: Churchill Livingstone, 1997.
29. Feinstein AR, Horwitz RI. Problems in the ‘evidence’ of ‘Evidence-Based Medicine’. American Journal of Medicine. 1997; 103: 529-35.
