The wartime generation of statisticians

If you haven't already seen it there is a great article in The Economist about the generation of British statisticians that cut their intellectual teeth during World War II.

Random REF thought

I can't help think that the REF has a number of features in common with the Deutsche Demokratische Republik.  

Few, apart from the apparatchiks with their snouts in the trough and those compromised by the Stasi  actually wanted it to carry on. Nobody really believed in it any more, but nobody wanted to say so. So everything just carried on until one day some brave people had the courage to say in public: "I don't believe in this any more" and amazingly they found that they were not alone. 

Ask a political sociologist. When legitimacy leaks away and people don't believe then it doesn't matter how many tanks are parked on the lawn. When the honest men and women massively outnumber the arse-lickers and lick-spittles then it is possible to say; "No, we are not going to take this any more, we want to do things differently".

REF 2014

So the waiting is over, the results are in and they are what they are. Congratulations to the winners and commiseration to the losers. As Brucie used to say "Good game, good game! 

A few weeks ago I posted on the  Mrygold, Kenna, Holovatch and Berche (MKHB) predictions for the Sociology sub-panel 2014 rankings based on the Hirsch (H) index. Just to remind you, the H index basically trades off citations against volume of production. So, it is no use producing a lot that nobody cares enough about to cite. To do well on H you have to produce work that people read.

So how did their predictions work out? Here's a graph I quickly ran off this morning. It plots 2014 REF GPA against  MKHB's H index score. Looks like an impressive relationship there, but in fact the Pearson correlation is only 0.15 (the Spearman rank correlation, to be fair is 0.65). 

What is  interesting is who is above and below the prediction line. The top 4 according to the 2014 REF ranking, York, Manchester, Cardiff and Lancaster, as well as Essex who come in at number 7 are all punching above their weight in H index terms. Another way of putting it is that the panel rated their outputs more highly than the research community (assuming that citation reflects, in the main, positive appreciation, significance, impact etc etc). Then there is the group that did less well than their H index suggests they should have, the OU, Warwick, Sussex, Brunel, Leicester and Queen's. If there is to be great wailing and gnashing of teeth then there is some justification for it from these guys.

Looking at this picture what strikes me most is how the H index really brings out three clusters of institutions: 1. Oxford, Manchester, Edinburgh, LSE and Cambridge where broadly speaking the H index and the REF evaluations agree in rating the institutions highly; 2. City, Goldsmiths, Manchester Met., East London and Roehampton where H index and REF agree in rating the institutions (relatively) poorly; 3. The crap shoot in the middle where the H index rates everyone about the same and where whatever it is that the REF panel members are thinking about , trading off and  higgling over makes all the difference. It would be really nice to know what that was, but I guess nobody is telling...

Other snippets of information that may be worth knowing: 

The Pearson correlation of REF GPA with number of staff submitted is 0.11  but the rank correlation is 0.60 ie  roughly the same as  with the H index. Having at least one  member of the REF panel from your institution is also correlated (modestly) with GPA (Pearson = 0.09, Spearman = 0.52).  And if you want to predict REF GPA without any direct measure of research quality then the  way to go is to use number of staff submitted plus whether you have a REF sub-panel member. The multiple correlation with these two measures is 0.19 ie you get a better prediction from this than from knowing the institution's H index score. Now that is food for thought.

So champagne for some and sack-cloth and ashes for others. But actually we are all losers from this ridiculous  and demeaning process. It's time for those who have come out of it smelling of roses (this time) to stand up in solidarity with those who have the faint whiff of the farmyard about them. There but for the grace of God etc.

And by the way, casting an eye over the rankings in a few cognate disciplines makes me think wtf!...

Causality fascism

While in Turku I heard a term that was new to me "causality fascism". Actually, in the usage, there is an important distinction to make. 

It could be used to designate someone who believes that the only useful social scientific work to be done involves the rigorous identification of some sort of average treatment effect. This view is beginning to take hold in some of the social sciences  - political science seems to be particularly prone to colonization - and is clearly dumb, dumb, dumb. 

It's OK to have other objectives, as long as you are absolutely clear about what they are. What the balance should be between uncovering heterogeneity and serious causal analysis will depend on the state of knowledge in the field. Frankly, if we are still struggling to establish what the facts of the matter are, then it is  a little premature to put too much emphasis on causality.

On the other hand, if somebody starts to use words like "effect", "impact", "influence" etc rather than "differences", "heterogeneity" and so forth  and has no viable strategy to identify  real causal effects, then a little "causality fascism" is surely a good thing. In this context forcing people to really address what the numbers they estimate actually mean in terms of the relevant counterfactual gives some sort of protection against the propagation of bullshit.

All of which gives me an excellent excuse to link to the classic Seinfeld Soup Nazi.

With Lenin in Turku - and on social mobility in Britain

I'm just back from Turku where I was participating in an excellent workshop hosted (very generously)  by Jani Erola and Elina Kilpi-Jakonen as part of their INDIRECT project. We were kept pretty busy in the conference room  during the day and in other ways during the evening so I didn't have much time for sight-seeing until the morning of my departure when I took a quick look around the city.

Turku is a charming place with a lot of elegant Jugenstil buildings in the vicinity of the central market square. The Swedish influence is still evident and the Svenska Teater was advertising a forthcoming production of  Ronja Rövardotter. Love of Astrid Lingren is a part of North European culture that Britain doesn't really share. To be sure Pippi Långstrump is  known over here but that's about it. My own daughter's love of Astrid Lingren comes from her mother reading the stories to her in German and Ronja Raubertochter is one of her favourites.

My wandering took me to the Turku Art Museum - a fine building in Nordic Romantic style - where there was an  exhibition of the work of the controversial Icelandic collage artist Erro. I'd never seen anything by him before and the  juxtaposition of incongruous images - the People's Liberation Army marching through New Jersey - was amusing and at the same time slightly unsettling.

Just opposite the museum is a  bust of Lenin who apparently stayed in the building behind it in 1907 when he was on the run from the secret police. It doesn't say on the plaque how long he was there, but it turns out that it must have been less than five hours as he was pretty anxious to get on a boat to Stockholm - well you would if the alternative was a lengthy stay in Siberia. 

 And what was I doing there? Well I was giving a paper on long-term trends in social class mobility in the UK. If you take the care to assemble as much of the broadly comparable evidence as you can it turns out that (at least for men - hold the headline for women) there is a pretty convincing case for believing that over the last 50 years - and possibly longer - there has been a continuous decrease in baseline levels of association of very roughly 1% per year ie relative rates of social mobility have increased. Trimming the data and applying all sorts of data exclusions doesn't alter the story much. 

Social mobility crisis? What crisis? You can find the slides from my presentation here.