Sunday, 26 May 2013

"Extraordinary claims require extraordinary evidence" - with respect to the claim of intelligence decline since Victorian times


The phrase and practice of "extraordinary claims require extraordinary evidence" is one of those superficially-plausible statements which are untrue, and indeed damagingly false.


Of course, this does not mean we should believe anything anybody might say no matter how absurd - but in practice the whole thing hinges on the meaning of 'extraordinary'.

In practice:

1. The definition of an extraordinary claim is 'something I don't already believe'...


2. The definition of extraordinary evidence is that no amount of evidence will ever be enough to convince me of that.


People find all sorts of things extraordinary which are not.

Modern atheist intellectuals, for example, find the idea of God or gods to be extraordinary, and all kinds of other things such as souls, angels and demons, the Virgin Birth and the resurrection of Christ - plus all sorts of things like telepathy and prophetic dreams - despite that these are not extraordinary to the vast majority of people alive, and to almost nobody in the sweep of human history.

Claims of the reality of things may or may not be factually correct - in general, or specifically - but these are not extraordinary claims - and the 'evidence' for them is of exactly the same nature as for anything else.


In science, a major area which asks for extraordinary evidence is claims of group differences in hereditary personality and intelligence. Around about 1965, it was rather suddenly decided that the claim of heritable group differences was extraordinary, and extraordinary evidence was demanded - and fifty years of research later exactly the same demand is still being made.

Each piece of evidence in support of the supposedly-extraordinary claim of heritable group differences is put under a microscope to check for flaws - and of course flaws can be found - especially when these people don't ever turn the same microscope onto their own beliefs.

So, those who reject heritable group differences believe what they believe, and which (because they believe it) they do not regard as extraordinary - and accept any further evidence in support of this belief with just a cursory glance; but anything they don't want to believe is checked, inch by inch, under a microscope and - guess what? - is found to have flaws! Therefore they feel justified in rejecting it.


This pretends to be 'rigorous' ('Look at me - I'm a real scientist! I'm using a microscope!') but it is of course, that is phony science, politicized science.

Indeed, the practice of accepting supporting evidence at a glance while putting opposing evidence under a microscope is not just bad science - it is not science at all.


Yet even outside of politics this technique of differential rigour is an easy trap to fall-into - indeed there are very few who are exempt.

This is something to be guarded against.


When I was editor of Medical Hypotheses it was the last major non-peer reviewed journal in science - and its special value was that it had the potential to overcome the intrinsic bias of 'peers' to defend the existing paradigm by demanding 'extraordinary' evidence for any new work which disagreed with the prevailing paradigm.

But what if the evidence for the prevailing paradigm is weak?

In modern professional science there are many dominant but evidentially (and logically) weak research paradigms - but these are defended as tenaciously - or even more tenaciously, than coherent and strongly supported paradigms.


For example, in psychiatry there is a decades-established prevailing explanation of 'depression' in terms of neurotransmitter abnormalities and of treatment by 'antidepressants' which correct these presumed abnormalities.

This paradigm does not make any sense and there is no evidence to support it - yet 'everyone' believes it - and any alternative perspectives are (therefore?) treated as requiring 'extraordinary' levels of evidence, such as clear and unambiguous support from multiple multi-million-dollar, mega-randomized controlled trials. Hence the prevailing nonsense is impregnable.


Declaration of interest: all this stuff is happening, now, to me and Michael A Woodley et al with respect to the use of longitudinal reaction time data to estimate long term trends in general intelligence. 

The problems are that the result contradicts current notions of:

1. Direction 

2. Size

3. Rapidity

of change in intelligence. Therefore the result challenges the current paradigm in several respects.


To put it another way, Woodley and myself are regarded as making an extraordinary claim - and this is used to justify putting this particular study under an extra-powerful methodological microscope (and if that doesn't suffice - an electron microscope) to search for, and inevitably find, microscopic cracks (gaps) and flaws (distortions) in the evidence.


But consider the counter-factuals.

Suppose that the data had shown a reduction in reaction time since the Victorian era (implying increasing intelligence) - which would be consistent with the Flynn effect - what would people have done?

In other words, what if the study - and I mean exactly the same study in terms of exactly the same evidence and methodology - had found exactly what people expected it to find?

Well, of course this would not have been an 'extraordinary' claim, so people would have accepted the study as a matter of course and without further discussion.


But suppose (again counter-factually) that the study (exactly the same data set and methodology) had instead found a small increase in reaction time? Implying a small reduction in average intelligence?

Well, that too would have been accepted without much discussion - since it would not signify much either way.

Nobody would have put the study under an electron microscope. 


But in real life the data showed a big increase in reaction times implying a big reduction of intelligence since Victorian times.

This is indeed a paradigm-shifting claim; and it is 'therefore' regarded as an extraordinary claim - and therefore this is used to justify (as described above) extraordinary examination of the claim - on the basis of the slogan in the title.

But is this an extraordinary claim?


If it really was extraordinary to claim that intelligence had declined by about one standard deviation since Victorian times, it would be easy to present many evidences and examples which very obviously refuted that claim.

Yet none have been presented.

Obviously, the best way to reject a claim is to refute it with clear and unambiguous contrary evidence - and if you are reduced to quibbling over methodology, then the claim is revealed as being not extraordinary - and therefore not requiring extraordinary levels of micro-critique.


So, although people feel that the claim of a rapid and significant decline in intelligence is so extraordinary as to invite extreme skepticism; in practice - or so it seems - it is not easy to refute this claim without stepping outside of real science and becoming the kind of phony, fake, pseudo-scientist who polices the field of heritable IQ group differences.

And this is what we find.


Under pretence of rigour, there is not just bad science, but non-science - anti-science.


So, if micro-analysis of potentially paradigm changing claims is revealed as anti-science, then what should be done?

Well, paradigm-changing claims should be evaluated in the same way and to the same standards of research competence and honesty as paradigm-supporting claims.

When a piece of research reaches the normal standards of competence and honesty, yet has extraordinary implications, then it is distorting and inappropriate and in fact anti-science to dwell upon the micro-details of this research.


The correct thing to do is to recognize that in real science multiple evidences converge upon the truth.

That is exactly why science is important - because true scientific claims have multiple consequences - or, to put it another way, the implications of important true theories ramify through reality, and therefore their consequences can be observed in many places.

So when the claim of reduced intelligence is based upon longitudinal change in reaction times, then the matter is not to be settled 'methodologically' by ever more, ever more voluminous and complex and precise measurements of reaction times - but instead by seeking convergent evidence from different fields - by tracing out the implications of the claim through reality until these implications go into places where they can be observed and checked.

Therefore, what is needed is to discover what would-be the implications of a significant and rapid decline in average intelligence, and then make observations to see whether these implications really have happened.


If the claim of an approximately one standard deviation decline in intelligence since Victorian times really is false; then, because this is a very significant claim, it should be easy to discover some strong evidence which contradicts the claim.

But such a claim need not, and certainly should not be required to, meet 'extraordinary' standards of evidence!


In conclusion:

If a claim is both 'extraordinary' and also wrong, it must be trivially easy to refute.

But if a claim which seems 'extraordinary' cannot in practice easily be refuted; then it is not really an extraordinary claim, and should not be treated as such. 


The Crow said...

Great Scott! You're an extraordinarily early-riser, Bruce.
That's admirable.

Meanwhile, I find that anybody who does not already know what I claim to be interesting fact, claims that what I claim is false.
I find this extraordinary.

Amusingly: as a child, I thought 'extraordinary' meant really, really, really ordinary.

Bruce Charlton said...

@Crow - Early? I force myself to stay in bed at least until 05.00h!

Wm Jas said...

Very good points.

This is a bit off-topic, but have you found increases in reaction time across races, or do you only have data for whites? If white IQs were once one SD higher, while black IQs have stayed about the same, that could explain why the Victorians were more "racist" than moderns; perhaps racial differences really used to be much more pronounced. (And the decline in RT could then be spun as a "good" thing, since it means the white-black achievement gap is closing!)

Bruce Charlton said...

The racial differences in reaction time are well known (written about by Jensen and Rushton for example), and consistent with IQ differences, but I don't know much about the detail and relative magnitudes - nor they match up with the longitudinal change.

To make a quantitative comparison would require the kind of matching/ controlling which Silverman (and Woodley) have done).

Wm Jas said...

Oops, I meant to write "the increase in RT," not "the decline."

dearieme said...

Here's what I commented on WUWT just the other day.

I must say that ““Extraordinary claims require extraordinary evidence” is drivel. What extraordinary evidence did Einstein’s claims need for support? One astronomical observation that may anyway not have been precise enough to do the job. What extraordinary evidence did Clerk-Maxwell’s field theory, or Planck’s quantum notion, need? Come to that, Newton’s claims about gravitation, the greatest advance in the history of physics, needed no new evidence to persuade people: it just cast a fresh and compelling light on heaps of old evidence.

Bruce Charlton said...

@d - Well said. (And considerably more concise than my post.)

ajb said...

"Extraordinary claims require extraordinary evidence" is usually just another way of saying "That seems difficult for me to believe."

The question is then: why is it difficult to believe? The evidential grounds are often quite slight, but then projected into a larger (supposed) evidential base.

dearieme said...

On the other hand, Climate Science may be viewed as a wonderful exemplar of "Implausible claims require implausible evidence" mixed with "bogus claims require bogus evidence".

Bruce Charlton said...

@ajb - "The question is then: why is it difficult to believe? "

Yes, that is a good question. Since people apparently seem to have no trouble believing that the 'incredible' increase in IQ test scores reflects and equally fast increase in in general intelligence - indeed people seem positively *keen* to believe this; then I suppose we are dealing with the idea of progress, including hopes of ever-increasing human and societal perfection.

Michael A Woodley has speculated that the 'transhumanism' idea - that is the idea of humans becoming (via technology) qualitatively enhanced in (for example) capabilities, lifespan - and the eradication of aging and pain - may covertly and implicitly be much more pervasive among HBD-interested intellectuals than generally acknowledged.

I think this may be correct - from my own experience. An optimistic atheist must locate his hope *somewhere* - and for many that hope is located in the enhancement of humans via technology.

Another strand of this is the New Age hope of human enhancement via evolution, higher forms of consciousness, a new kind of spirituality etc.

On The Left (which includes non-Religious conservatives and libertarians and fascist types) the official world view is that each generation transcends the previous one; and social, moral and other changes are justified because of our superiority to those in the past - for intellectuals, modern superiority is supposed to be intellectual, as well as moral.

Anyway, if such views/ aspirations are covertly and implicitly as widespread as I suspect; then it may explain why on the one hand evidence of rapidly increasing IQ is accepted as increasing intelligence, while on the other hand evidence of declining 'g' is regarded with hostility and suspicion.

This doesn't help settle which, if any, of these views is correct - but it may somewhat explain the differential reception of ideas.

imnobody said...


I don't know why you are so cautious in your last comment. The myth of progress is one of the dogmas of our time. This is so obvious that does not merit discussion.

Everything that goes against this myth is considered suspicious. That's obvious too.

Anonymous said...

Forgive me for being relatively unfamiliar with this topic...

This claim of decreasing g, while it may not convince everyone, is extremely important, and needs further study. It must not be dismissed merely because it runs counter to accepted wisdom.

How about reaction time changes in China and other far eastern nations? How about America? Russia? Has the drop been steady, or is it accelerating, or leveling off?

What about places where infany mortality has been very high until recently?

What are the details in how the reaction times were measured in the 19th century vs. now?

Bruce Charlton said...

@ANON - I would guess that the best bet for finding another, independent, source of old reaction time data would be in the German language scientific literature of the 1800s - since Germany was at that time the leading nation in psychological/ physiological research.