Oxford University = University of Minnesota

What has the RAE ever done for Oxford University’? Improvement in normal science – but decline in revolutionary science.

Bruce G Charlton

Oxford Magazine - 2008; 271: 3-5

Oxford has done well in the past UK national Research Assessment Exercises, and will probably do well in the current and any future RAEs. But what good has the RAE done Oxford, and the UK university system?

I think the answer is that the RAE has done some good and also some harm to UK research universities, with perhaps the good outweighing the harm in most places – but in Oxford (and also Cambridge) the damage from the RAE outweighs the benefit.

This is because the RAE has probably increased overall science production at Oxford and elsewhere; and this is good. But the RAE has also measurably reduced the production of the very highest quality science; and this is bad for Oxford. Oxford (and Cambridge) - more than other UK universities – used-to specialize in producing the highest quality of ‘revolutionary’ science. Not any more…


Revolutionary science compared with normal science.

Oxford and Cambridge are bigger and more productive than other UK universities – their science output is each about twice as big as the third and fourth most productive universities which are Imperial College and UCL (Oxford Magazine 2006; 254: 19-20). However, Oxbridge is no longer qualitatively superior to other UK universities in the way that it used to be. True, Oxbridge still produces more research than other places; but it is now hard to see any objective evidence that Oxbridge produces better-quality science than elsewhere.

Taking the terminology of Thomas Kuhn; ‘revolutionary’ science which changes the direction of science can be distinguished from ‘normal’ science which is an incremental extrapolation of established science. I suggest that Oxford has declined as a centre of revolutionary science at the same time as it has improved its production rate of normal science – and that the RAE has something to do with this.

In terms of its total number of publications and the citations to those publications, Oxford has been catching-up with the best US universities over the past ten to fifteen years. Indeed, this has been the pattern for all the best UK research universities. With Peter Andras (Medical Hypotheses, forthcoming) I have demonstrated that the average UK top-twenty university has improved its Web of Science ranking in terms of annual total citations from 83rd to 65th from 1990-94 to 2000-4.

This marks a change of trend, which was probably triggered by the RAE. Looking back over 30 years, the UK was somewhat declining relative to the USA up until about 1990, but since then has been catching-up. Overall, the pattern of data is consistent with the RAE having progressively increased the number both of publications and also the citations to those publications.

Sir David King, who was Chief Scientific Officer, calculated that from 1993-7 to 1997-2001 the percentage UK share of total world publications was second only to the USA and increased from 9.29 up to 9.43 while citations increased from 10.87 up to 11.39 (Nature 2004; 430: 311-16). At the same time, US percentage shares were declining for both publications (37.46 down to 34.86) and citations (52.3 down to 49.43). So it is plausible that the RAE should be credited for helping increase the total production of UK science more rapidly than it has increased in the USA.

In other words, the RAE has probably led to an improvement in UK normal science.


The Oxford (and UK) decline in revolutionary science


Nobel prize data

However, even as it improved production of normal science, I suspect that the RAE has simultaneously led to a decline in UK revolutionary science. My measures of revolutionary science come from two sources: science Nobel prizes and the number of ISI Highly Cited (HiCi) academics.

Nobel prizes are usually awarded for revolutionary science. I have looked at the trends in science Nobel prizes over the past 60 years (data available at http://medicalhypotheses.blogspot.com/2007/07/nobel-prize-trends-19472006.html) and it makes depressing reading for a UK scientist.

The UK has dropped from a clear second place to the USA in terms of total number of laureates, with per capita more Nobel prizes than the USA – to the present situation in which the UK won only 9 prizes from 1987-2006 compared with 126 in the USA. Five UK-born laureates had emigrated to the USA before winning prizes – which is evidence of the UK’s inability to hold-onto its very best scientists (by contrast, for 60 years, no US-born laureates have ever moved to the UK).

Oxford won 3 Nobel prizes 1947-66, another 3 Nobel prizes 1967-86 – but none since. In the past 20 years Cambridge managed 2 prizes. But MIT won 11 prizes, Stanford won 9, Columbia and Chicago 7, Princeton 6, Harvard 5… In the US public universities Berkeley got 4; UCSF, Irvine and Santa Barbara 3 each; Colorado at Boulder 4 and Washington (Seattle) 3. The message seems clear - the US university system is now totally out-performing Oxbridge in terms of science Nobel prizes – and (I suggest) probably therefore in revolutionary science in general.


Highly-cited (HiCi) academics

However, there are only a maximum of 12 Nobel science laureates each year (including economics) so Nobel prize data is statistically imprecise as a measure of revolutionary science. However, Nobel data can be amplified using ISI Highly Cited (HiCi) academics. HiCi academics are calculated by Thomson scientific from the Web of Science database to identify the researchers who have the greatest impact on their specific fields in terms of being cited by other researchers in the field. Most Nobel prize-winners are drawn from the ranks of HiCi academics, so it seems reasonable to use HiCi status as a measure of revolutionary science. However, some HiCi academics are probably more accurately regarded as very productive and highly-respected ‘normal scientists’.

Universities can be studied in terms of the numbers of HiCi academics on their faculty. My assumption is that HiCi faculty are a measure of the amount of revolutionary science at a university.

Using the current Thomson Scientific database (http://hcr3.isiknowledge.com/formBrowse.cgi) it can be seen that Oxford and Cambridge top the UK rankings again with 45 HiCi at Oxford and 52 at Cambridge. For comparison UCL has 24, Imperial College has 31, Bristol and Manchester 15 each. The number of HiCi academics among these UK universities broadly correlates with the total volume of annual publications and citations in Web of Science.

But when we look at the US universities it brings Oxbridge’s performance into a startling perspective. It is easy to lose count – but I think Harvard has about 185 HiCi. Even leaving aside the US private sector universities, Berkeley has 86 HiCi, and the University of California San Diego 60 HiCi, and University of Washington at Seattle 52.


Oxford equals Minnesota

So if we were to make an objective comparison of Oxford’s research performance – and tried to place Oxford among US universities using scientometric data such as total citations, the number of Highly Cited academics and Nobel prizes – what would be the nearest matching university?

Somewhat shockingly, the closest match to Oxford in research terms seems to be the University of Minnesota.

Oxford and Minnesota have the same number of citations for 2000-2004 (others at this level of production are the University of Pittsburgh and Duke); Oxford currently has 45 HiCi academics compared with 47 in Minnesota, again an almost exact match (Pittsburgh manages 31 HiCi and Duke 40). And neither Oxford nor Minnesota are winning significant numbers of Nobel prizes in science (Minnesota got its first one, for Economics, in 2007 but Oxford’s most recent laureate was 1973). So, both Oxford and Minnesota are big and successful universities mainly specializing in normal science.

While Oxford is justifiably proud of its distinguished history and current international name recognition; in terms of objective scientometric comparisons Oxford is currently equivalent to the University of Minnesota. Does this matter? Well… on the one hand Minnesota is a very good university by international standards. On the other hand it is probably only in the third tier of US universities, while Oxford is generally considered to one of the elite universities in the world. The fact is that Oxford nowadays is not one of the elite universities of the world in terms of scientific research. It used to be, but not any more.

Naturally the all-round academic strength of Oxford extends beyond the mostly scientific subjects which are measured using scientometrics. In particular Oxford is very strong in the arts and humanities, and probably publishes more in this area than any other university in the world (according to the Web of Science database – Oxford Magazine 2006; 256: 25-6). Furthermore, Oxford probably provides a much better undergraduate education than most other big research universities.

However, in a global system the status of universities will increasingly depend on their objectively measurable scientific research performance. Arts and humanities are culture-specific, while undergraduate teaching is notoriously hard to compare (is Oxford really better than Chicago, or small liberal arts colleges like Amherst and Wellesley?) Since a university can only prove its superiority in science, science is what is used to measure status. And in terms of science Oxford is no longer especially distinguished – neither in volume nor in quality. There is no room for complacency. Europe is littered with once-great but now merely-moderate universities – Halle, Gottingen, Berlin, Heidelberg, the Sorbonne, Bologna, Pisa, Salamanca… I suggest that Oxford may be on this path.


The pressure to down-shift from revolutionary to normal science

So - in terms of normal science, Oxford is improving – unfortunately this improvement seems to be at the expense of revolutionary science.

My explanation for these trends in opposite directions is that current incentives favour ‘down-shifting’ among the very best academics. Oxford is a highly-attractive and very selective institution; so it seems reasonable to assume that Oxford contains scores of individual scientists who are capable (with luck) of doing top quality revolutionary science (at the kind of level that might potentially win a Nobel prize). Why are these best scientists at Oxford no longer producing revolutionary science of the highest quality?

The matter is complex, but my hunch is that these top-notch Oxford scientists are not being encouraged to do the best work of which they are capable. They are not being encouraged to tackle the biggest scientific problems which they have a chance of solving.

Indeed, it is worse that mere lack of encouragement to do top quality scientific work, it is a matter of positive pressure to down-shift to second-class work. Maybe they are not even being allowed to take the risks entailed by aiming high?

Instead of being urged to take on tough problems where there is a significant chance of failure, the best young Oxford scientists are being pressurized (implicitly for sure, but probably sometimes explicitly) to do easier, more predictable, and more short-term research. Why? Because this is the kind of research which has a higher probability of getting funding, and leading to large numbers of well-cited papers; and all in a timeframe so as to be ready for the next up-coming RAE - with the extra money this brings-in.

Oxford’s success in expanding normal science and performing so well in the RAE may therefore have been achieved by sacrificing its performance in revolutionary science. To put it bluntly: Oxford looks as if it is taking some of the best young scientists in the world – scientists who have the potential to make revolutionary contributions to their subject – and Oxford is converting them into normal scientists – albeit highly-professional and unusually productive normal scientists. After they have down-shifted their ambitions they become superbly effective at winning big research grants and generating a large volume of well-respected papers, and this work is both necessary and valuable; but it looks like nowadays most of Oxford’s scientists are not even trying to solve big problems.

The RAE is almost certain to make this down-shifting more prevalent and more severe – because if a scientist chooses to aim-high this entails an inevitable sacrifice of short term productivity and funding; as well as a much greater chance of failure. A truly ambitious scientist may be prepared to risk his or her career in trying to solve a really important but tough problem – however if he or she does so then s/he may feel (or be perceived as) selfish.

Helping the cost centre achieve a high RAE score could mean hundreds of thousands of pounds extra money per year. Failing to optimise short term productivity implies the possible loss of hundreds of thousands per year. Scientific idealism then entails financial penalties – with results that many include sacrificing the department, colleagues, research teams jobs - all for the (perhaps delusional) hope of a paradigm-shattering scientific breakthrough. How selfish of them.

Most universities do not have many top-quality scientists – capable of doing work at the highest level - but Oxford does. However, it looks like Oxford is either failing to bring out their potential or (worse) actually thwarting their potential.


Conclusion

The government is prone to self-congratulation regarding the RAE, and I think it only fair to acknowledge that the RAE deserves credit for making UK science measurably more competitive and driving-up total production relative to the USA. Perhaps, on the whole, the RAE may even have benefited most of the researchers in most of the most research-active universities.

But equally, I think that supporters of the RAE need to acknowledge that there is objective evidence of significant decline in UK revolutionary science, and that this can also plausibly be linked to the RAE. For most universities this does not much matter, because they never did have much first class research going-on. But for Oxbridge this down-shifting has been lethal.

All this points to the surprising conclusion that – despite their unmatched success in winning RAE-linked funding - the RAE has probably damaged Oxford and Cambridge more than it has helped them – since the RAE has presided over the decline of Oxbridge from their elite status of being centres of elite revolutionary science.

Of course, it would be simplistic to blame the RAE alone for the decline of UK and Oxford revolutionary science. The RAE is merely the tip of an iceberg of UK government regulation which has made universities ever-more short-termist. And even assuming the RAE is indeed a major contributor to this decline, the RAE would never have been introduced in the first place nor would it have survived if there had been a strong culture of revolutionary science, or strong incentives to encourage revolutionary science. And of course the government must take the primary blame (as well as credit) for the consequences of the RAE.

A major (probably unintended) effect of the RAE on Oxford has been to make Oxford more like other UK universities in terms of the type and quality of research. Scientifically-speaking, Oxford looks like a bigger, more-productive version of the other Russell Group universities like UCL, Manchester and Bristol – different in production volume but not different in kind.

For Oxford again to become a major centre of revolutionary science would be a big ask. It would require a significant reining-in of the influence of the RAE. But it would also require more.

Oxford would need to recognize the primacy of science in international university comparisons, and also recognise the primacy of revolutionary science within science. Increased quantity does not compensate for decline at the level of highest quality. Oxford needs to lose a certain smugness in relation to its currently undeserved international reputation – because even anciently-established international reputations will eventually adjust-down to the level of performance.

As a corrective to wishful thinking may I suggest that the all-too-common Oxford daydream of being as-good-as Harvard, Stanford and Berkeley should be tempered with the sober reality of actually being as-good-as the University of Minnesota; and cheerful daydreams of ‘closing the gap’ with the US elite need to be supplemented by ‘nightmares’ about the distinct possibility of Oxford becoming as-mediocre-as Berlin, Bologna or the Sorbonne.

Bruce G Charlton teaches at Newcastle University, UK; and is the editor of Medical Hypotheses.