Scientific papers generally have sections on methodology, evidence, modes of measurements, actual data, discussions and conclusions. The aim is to be non authoritative, to explain why these results should be judged to be true, and accurate, so they can be tested against experience. A quotation attributed in some form to Galileo and Einstein is: `Authority is the enemy of true science.'
It is curious, that, by contrast, the RAE embeds `authority' at the centre of assessment. It assumes there is a well understood notion of `excellence', and that a few chosen people are well placed to determine this, with little hint of the methodology.
Their judgements are given as if ex cathedra, contrary to long standing academic traditions, which were developed for good reasons. Also, the `excellence' determined is that of departments, or research units, not of individuals, except collectively. Yet famous people in science, such as Francis Crick, have urged that research is done by individuals. All those in a high rated department get much more funding than all those in the next rated department. There is, it seems, no mode for the RAE to support talented individuals, or exploration for its own sake, a common method of progress in science.
Further, what is a `research assessment unit', in this internet age? The question is not addressed, it seems! So much for looking to the future! Perhaps the practicalities of assessment over-ride any theoretical foundation?
This practicality problem arises also with the notion of Assessment Unit, in the Internet Age. Such a Unit is strongly associated with an address and a University. But nowadays mathematicians are more likely to collaborate with someone literally over seas than with someone down the corridor. Companies are formed with people working on their home computers. The RAE model of what is research activity may already be years out of date, and is now fixed till 2008! The word `Internet' does not occur in the RAE Gudance!
The RAE gives its criteria in terms of `national excellence' as against
`international excellence'. Does this imply
the UK is determined by our masters to be second rate?
The glory of mathematics and science is their
independence of geographic, national, and political
boundaries! It is well said that `national science is non science'.
In 40 years of refereeing and reviewing, I have been asked always `how good
is the work?', but not: `is it of national or international excellence?'
A phrase used recently in a newspaper in regard to British culture was :`it
shows a cringeing respect for all things foreign'. How can the UK develop
new science unless scientists are encouraged to explore new, independent,
lines, those not under strong international investigation? The word `exploration'
does not occur in the RAE Guidance!
The Panels do not ask for information on what new lines have been started? or developed? on what explorations of possible new worlds have been undertaken? Without such explorations, we are living on capital, on our reserves.
I find no evidence that those who work out these assesment systems, and possibly
also those who apply them on the Panels, have read nothing on the progress
of science, such as the work of
Thomas Kuhn
on the `Structure of Scientific Revolutions'. There are homilies such as:
`Great oaks from little acorns grow.'
`Rome was not built in a day.'
`Scientific paradigms can go down as well as up.'
Do people trust an investment company which assures potential investors it
will always `choose the best investment'? One might ask for information on:
`How will they do that?' Research is a risky business, it is not a form of
engineering, applying standard tools to already formulated problems.
A young ambitious worker in a small department would find it difficult to
challenge the big departments on their own grounds, for many reasons. So
such a person will look for anomalies, for the questions undiscussed, the
doubts not raised, for techniques and ideas unexploited, and will make a
judgement on how to proceed, and the risks involved. It is much nore fun
to pursue novel ideas, to raise new questions, than to try to solve
already formulated problems! Does experience show that such attempts by those
outside their groups are appreciated by the `big guys'? What happens
to those who try to disclose new worlds? Does
`Flatland' have
an answer? Professor Alan L. Mackay, FRS, described, in a lecture at Imperial
College in 1985, the history of icosahedral symmetry in crystals:
`First they said it is not true. Next, they said it
is true, but unimportant. Eventually, they said: It is true, it is very
important, and we have known it for years!'
The RAE Guidance document does not contain the word `pioneering' and it contains the word `innovation' once only, and that with regard to `technical'. The word `excellence' is very well used - once again, Orwell's comments on the impoverishment of language are relevant. How will pioneering and innovative scientific departures take place in the UK under this financial reward system?
The RAE also does not use the word `communication'. But this is vital for interdisciplinary research, particularly with regard to mathematics. My experience in trying to get over mathematical ideas to the general public and to13 year olds (see www.popmath.org.uk) has been essential in giving presentations both to interdisciplinary audiences and also even to mathematicians, using the same material in each case with slightly different emphases! A wide range of people wnat to know about `advnced mathematics from an elementary viewpoint'. In any case, good communication has been proved to be important for the progress of mathematics, since it leads to increased understanding. It is not a luxury, tagged on at the end! Euclid, Gallileo, Klein, Poincare, Einstein, Feynman, .. all spent considerable effort on communication. However, the UK has decided to discount their lead, in the interests, so it is claimed, of helping the UK to become world leaders! World followers, more likely!
There is no place in the RAE assessment for long term projects, since the Panels have a 4 or 5 year time view. Long term projects go up and down in their publication profiles, as major milestones are reached, followed by consolidation. Initially, such projects may seem small time, far from `mainstream'. Even `mainstream' is shown by experience to flap around like a flag in a storm (to mix a metaphor)! Big research themes rarely come like Venus Anodyamene, fully formed from the sea, though an individual (not a committee) may have such a vision. The Panels find it easiest to judge `excellence' by publication in `top journals', which are of course run by the `top people', and are geared towards the current paradigms. Thus does authority propagate itself!
The RAE Panels sometimes behave like Marie Antoniette: `Why do they not eat cake?' There is a Catch 22 situation: they ask for plans such as replacements of `a key figure', which maybe the University cannot support without the funding and approval of a good research rating, but that rating is dependent on guarranteed plans and support for a replacement! (This has occured at least twice in the last assessment.) The words `value for money' do not occur in the Guidance.
There is also the actual assessment report. This can read like a school report:
`Could do better.' In view of the expense of the RAE, one could also
recall:
`The Doctors said as they took their fees
There is no cure for this disease.'
I like the radical medical view:
`Diagnosis without treatment is unethical.'
Einstein said: "To punish me for my contempt for authority, fate has made me an authority myself.." As an authority, he told de Sitter that the notion of an expanding universe was `bad physics'. In the current UK regime, that would surely have been enough to close de Sitter's department!
A report on a Bangor research proposal in 1989 wrote: "There were substantial reservations, however, about how interestingly or significant the results would be outside your immediate school, as the impact of this type of work does not appear to have been widely felt so far, in spite of the quite impressive publication record cited." I thought we were asking for funds to help develop this new world we had found! We had major publications in 1981-89, involving collaborations with Philip Higgins, Edmund Robertson, David Johnson, Jean-Louis Loday, among others, as well as a correspondence with Alexander Grothendieck, 1982-91, publicly described by him as `a baton rompu'! The time lag for influence, and recognition of the influence, of a new mathematical idea is long (cf., Galois, Grassmann, ...). Our first paper on the nonabelian tensor product of groups was published in 1984, and the bibliography on this topic has now reached 88 items. But this topic is only a glimpse of the world of multiple groupoids and their possible applications.
Groupoids were defined in 1926, and my book of 1968 emphasised their utility in 1-dimensional homotopy theory, convincingly, as I thought. Yet this idea has not been taken up generally in texts in algebraic topology. Abstract groups are currently much more important in the scientific community than abstract groupoids. How long will this last? Subsequent work on higher dimensional analogues of groupoids has proved difficult, as is most exploration, and not too easy to take up, since there are, apparently, few easy pickings (what have I missed?). Lie groupoids are becoming increasingly important, since the pioneering work of Ehresmann and Pradines.
Perhaps also the fact that a research proposal is judged on sociological rather than mathematical grounds reflects on the lack of a serious development of what one might call `mathematical criticism' or `analysis of methodology', of what constitutes `good mathematics'. So there is left the naive analysis: `Good mathematics is what the top people do!'. This type of view has been strongly attacked by Grothendieck in his long but unpublished book: `Recolte et Semaille'. David Corfield has discussed related issues in a recent book.
The history of science and mathematics shows that many major developments have arisen from young and ambitious people working on their own, and following through an idea unappreciated by the established, and therefore ripe for the picking. A new idea does not get published in the `top' journals, or attract research grants from Research Coucils, unless it immediately fits with their current paradigms, and therefore is less original! This aspect of the dynamics of research progress seems foreign to the RAE. Anyone who doubts this aspect should read books by Lovelock on `Gaia', or consider the history of Wegener's theory of `continental drift'. It is surely too cynical to suggest that the RAE replace the word `excellence' by the word `kosher', but that does suggest that the methodology of research assessment, as far as it can be determined, is related to that used in the past to judge heresies.
Here is a quotation from G.-C. Rota (for which I am grateful to David Corfield):
"What can you prove with exterior algebra that you cannot prove without
it?" Whenever you hear this question raised about some new piece of
mathematics, be assured that you are likely to be in the presence of
something important. In my time, I have heard it repeated for random
variables, Laurent Schwartz' theory of distributions, ideles and
Grothendieck's schemes, to mention only a few. A proper retort might be:
"You are right. There is nothing in yesterday's mathematics that could not
also be proved without it. Exterior algebra is not meant to prove old facts,
it is meant to disclose a new world. Disclosing new worlds is as worthwhile
a mathematical enterprise as proving old conjectures."
(Indiscrete Thoughts, p.48)
Which of these last will be more highly rated in a Research Assessment? Which is easiest to `measure'?
Certainly two successive RAEs have not rated as internationally excellent our work in `Higher dimensional algebra', which in UK mathematics (as against computer science) is currently studied outside Bangor mainly, but not entirely, at Cambridge. Some outside the UK see this topic as a potential major mathematical theme of the 21st century! See the counters on the above site for the international interest (over 17,200 page views since May, 2000), and do a web search for other applications.
Is the RAE patterned so as to reduce diversity? to deter exploration,and pioneering work? to stop individuals developing new areas? Does it seek value for money? Where is the comparative monitoring of this? What is the evidence for its methodology? It is notable that there are no routes for debate.
The idea of funding research according to measurements of qualities is not new. Sir Peter Swinnerton Dwyer was asked about this at a meeting of Heads of Departments of Mathematics in the 1980s when he was Chairman of the University Grants Committee. He replied that there was a question not only of the amount that should be allocated according to this judgement, but even of its sign!
A radical medical view is: `Diagnosis without treatment is unethical.' The RAE Panels are not there to help and advise, but solely to assess. Yet they regulalrly bring in new criteria! A recent one is `sustainability and vitality'. Of course `sustainability' is to some extent in the RAE's disposal. They do not ask if the Assessment Unit has a long term record of a sustained research programme. The meaning of``vitality' is unclear.
The current methodology realises the old adage: `The best is the enemy of the good.' This is seen in the closures of good departments in the UK, for `inadequate' research attainments, as judged by by RAE Panels, whose judgements include no justification, no explanation of methodology or context, and no appeal. To what in wider terms should this situation be compared? It is certainly an emphasis, which is generally supposed to date to prescientific eras, on AUTHORITY as a control of research!
In mathematics, one can believe that Galois, Cantor, Grassman, ... , would have done badly in any supposed contemporary RAE. In the work of Heinrich Brandt, we have a 6 page 1926 paper on groupoid as a `generalisation of the notion of group'. Even now, and despite the efforts of myself and others, groupoids are not accepted by the establishment' (e.g. the members of the RAE Panel in Pure Mathematics) as the natural context for many ideas on symmetry. But this paper is growing in influence. Who would have forseen this?
The main problem is perhaps that Government wants to save money, so it would like to be seen to invest in the research which will be world leading and have the most impact. The question which seems not to be put is whether such a method of investing in `world leading research' is likely to work. Evidence across a number of fields suggests that a broad base is likely to be needed to produce champions. In rose breeding, I am informed that `usually one seedling in 10,000 brought to budding stage will be worthy of further study. If it refers to exhibiting, then you will need several plants of each variety you want to show.'
Part of a report of the Engineering Professors' Council meeting with Barry Sherman MP, Chairman of the Education and Skills Select Committee
"On the Effects of the RAE and its impact on University management he seemed a little surprised at the EPC comments on the impact of the RAE exercise and the resultant emphasis on research with its consequent downgrading of teaching. He felt that the survey on student satisfaction might go some way to redress the balance. Although he felt that it would not be possible for his Committee to have an enquiry into the effects of the RAE, he did raise the possibility of organising an evidence session where the committee could question a number of VCs and the staff from HEFCE who would be running the RAE."
IMPROVEMENTS?
What do these arguments suggest is necessary to `improve' the RAE, in the sense of making it less damaging to the future of science in the UK? There have been some improvements for the 2008 RAE, in that international members of Panels have been brought in, and the panels have been broadened. This should make the distinction between `recognised as internationally excellent by the Panel' and `internationally recognised as excellent' less severe. On the other hand, it presumably makes the whole affair more expensive. The notion of `value for money' does not occur in assessment of Research Units nor of the whole exercise.
The major improvement is to make the funding change across grades less severe. At present, the procedure is `unfriendly' in the sense of computer programs: a small error of judgement in presentation to the Panel can have, probably have had, consequences of life or death for a department. Of course such a financial change will not happen, as the `top' departments and Universities want to hang on to their funding. They will fight in all possible ways to retain their privilege, and argue that any change will be the end of the world as we know it!
It is often argued that a democracy and free discussion is necessary for invention and progress. If so, this does not auger well for the progress of science in an oligarchy and autocracy.
January 30, 2006
Return to Popularisation and teaching page
Return to home page