Popular Posts

Caveat Emptor

The opinions expressed on this page are mine alone. Any similarities to the views of my employer are completely coincidental.

Monday, 5 November 2012

Open letter to Prof. Richard Wilkinson & Prof. Kate Pickett

Dear Professors Wilkinson and Pickett

You don't know me and are unlikely to read this letter. Even if you do, I know  you won't reply to it because you have stated publicly that you will only reply to criticisms of The Spirit Level that are published in peer reviewed journals. I think that is a mistake - what matters is the cogency and import of the criticism not where it is published - but I fully understand that you are busy and important persons, that life is short and that one has to draw the line somewhere. 
In any case I'm probably even less worth replying to than your average critic: I'm not an epidemiologist, public health expert, statistician or think-tank pundit and I have never written on public health matters for peer reviewed journals. I'm merely a  sociologist who has spent  most of  his professional life trying to make inferences from quantitative data about the structure and process of social life  while teaching his students, to the best of his ability, how to interpret numbers in sensible ways. It's a small thing but it is my own. 
I should probably also say, though strictly speaking it should be irrelevant, that I don't identify myself with the political right. In confessional mood, I admit that though I've voted for several political parties, I've always voted for parties that favour more rather than less redistribution from the relatively rich to the relatively poor and that I'm an unabashed admirer of the social and cultural arrangements found in some of  the Nordic social democracies. If one finds labels helpful I'm more Old than New Labour though I don't happen to be a member of any political party and like Keynes I'm not ashamed to change my opinions when the facts change.
Normally I would regard  wearing my political heart on my sleeve as unforgivable bad taste, but in this case I think, for the sake of clarity, it is necessary. It's easy to dismiss critics as ideologically motivated and use that as an excuse not to engage fairly or indeed at all with the points they raise. Though it is presumptuous to say so, it seems to me likely that part of the motivation of the critics that you have found the time to reply to - Christopher Snowdon, Peter Saunders and the Tax Payers Alliance - is indeed ideological. Nevertheless, as I constantly remind my students, the provenance of an idea is completely irrelevant to the evaluation of its truth and I must confess that I find many, if not all, of the points raised by this trio reasonably argued, deserving of serious attention and, in the cases where they are factually wrong, careful point by point refutation. I'm not sure that you have always held yourselves to this ideal standard, but then in the rough, tough battle of ideas, and with human nature being what it is, I'm probably just naive to expect scientists to welcome stringent assaults upon their ideas with truly Popperian equanimity.
By now you are probably impatient to learn what this is all about and I beg your indulgence for just a little longer while I explain. I thought it would be interesting to ask my students to read The Spirit Level. They are all bright graduates with a keen interest in public affairs and, given the nature of my department, a keen interest in learning how to evaluate quantitative social scientific research. They are assuredly not your average group of sociology students, and all the better for it. In order to keep our seminar within the bounds of the allotted two hours I asked them to focus exclusively on the sections relating to physical health and assigned them a number of review and commentary articles as supplementary reading. I realize that your book is about so much more than just physical health but I feared that if we were to attempt to discuss all the domains that you write about our conversation would  become unfocussed, inconclusive and pedagogically useless. In any case, unless I have seriously misunderstood you, I take it that evidence about physical health is actually central to the claims you wish to advance.
My first task was to clarify exactly what those claims are so that the students could home in on the precise evidence that is most likely to speak to them. Again I have to ask for indulgence. As an educator I'm wont to simplify things and in my quest for pedagogical clarity I may be prone to over-simplification. I hope what I said is not  so gross a distortion of  the case you wish to make as to make what I wish to say irrelevant to your concerns.
My pitch to the students was as follows. Nobody of any repute disputes the fact that for many health related outcomes there is a soci-economic gradient. For simplicity let's just say that at the individual level income predicts mortality risk. The relatively rich live longer and the relatively poor die earlier. It is plausible to believe that this predictive relationship, which may or may not be interpreted causally, is non linear. If a cut purse takes ten pounds out of the pocket of a rich man he probably won't notice and it is implausible to believe that his loss will have much, if any, discernible effect on his mortality risk. On the other hand if Robin Hood gives ten pounds to a poor man on the verge of starvation it makes quite a big difference - he lives to fight another day. This is nothing more than the standard welfare argument for equality. Up to a point, distributing income away from the rich towards the poor is likely to increase aggregate welfare because dw/dy ~=c. Whether this is true or not is ultimately an empirical issue albeit one that it is quite difficult to produce clinching evidence about, but like me I imagine you believe it is likely to be true.
The key point, for our purposes, is that this has nothing to do with the aggregate level of inequality in the society in which the rich and poor man live. If we want to know, what the direct contextual effect of aggregate level income inequality is on individual level mortality risk then we must correctly specify the relationship between income and mortality risk at the individual level. Otherwise part, though probably not all, of the apparent relationship at the aggregate level turns out to be spurious - a form of the ecological fallacy. Of course you know this and I apologize  if  in my eagerness to make things clear to my students I seem to be teaching granny to suck eggs.
The take home message to me is clear: if  the relationship at the individual level between mortality risk and income is non linear there is quite a lot of good we can do to improve the health of the least well off portions of society by spending money in ways that reduce the exposure of the poor to well known risk factors. The net effect of this, depending on exactly where the money comes from, may well also be to produce a more equal society both in terms of money income  and  in terms of welfare which is what, after all,  we should actually care about. And all this could be achieved, I conjecture, without having to give much weight to a speculative theory of the psychosocial aetiology of illness. But more on that later.
Of course we live in an imperfect world and it is particularly imperfect, as you are well aware, in terms of the data resources we can use to evaluate our empirical claims about it. Given the impossibility of carrying out a randomized controlled trial - we can't randomly select a bunch of rich Americans, force them to live in Sweden, and wait and see if they live longer than the ones we left in the US - we are stuck with observational data. I believe, and I doubt this is contentious, that if we want to know whether aggregage level income inequality has a direct effect on individual level mortality then the ideal data source would be a multi-level time series with observations at both the individual and aggregate level within one nation state - to control for all the confounding unobservable cultural and institutional stuff that makes one country different from another - while allowing us to see what happens at the individual level (after controlling for individual level exposure to risk factors) when aggregate level income inequality changes. If we could solve the serious problems of data comparability there is no reason why we shouldn't throw more than one nation state into the pot, but let's not get our hopes up, getting good quality multi-level time series data for just one country is a big ask.
So we have to go with what we have got. What you have got comes in essentially two flavours: aggregate level data on a sample of rich countries, subject to various inclusion and exclusion criteria (which have been pretty thoroughly picked over by your critics) and aggregate level data on US states. You show us in Figure 1.1 (pp 7) that your scope conditions are defined by those countries from that part of a bivariate scatterplot of per capita national income by life expectancy where the increase in life expectancy for a unit increase in national income is close to zero. Your interpretation is that for this set of countries national income has no discernible effect on life expectancy. What it tells me is that beyond a certain level of affluence different countries have different cultural and institutional ways of producing roughly similar aggregate level health outcomes  (there is no one best way) and that it would be important to take that into account when trying to explain why some do (slightly) better than others.
Of course, like you, as a paid up social scientist I find the "everywhere is different from everywhere else" of the dyed in the wool comparativist a bit unsatisfying. We've imbibed with our mother's milk the idea that the objective of cross-national comparison is to replace the names of countries with the names of variables. So enter income inequality.
You show us in Figures 2.2 and 2.3 (pp 20-21) that among countries an index of health and social problems is quite strongly correlated with income inequality but only weakly correlated with per capita national income (though you must be aware that the regression slope, which you fail to include in Figure 2.3 but boldly highlight in Figure 2.2,  is quite strongly influenced by the outlying position of the US). You then go on in Figures 2.4 and 2.5 (pp 22) to show us an apparently similar pattern for US states, or at least that is what the legends of your figures suggest. Again you emphasize the strength of the relationship for income inequality by including a bold regression slope but exclude it from the per capita income graph. Let's assume that all this is within the bounds of  allowable rhetorical exaggeration. But wait a minute. My first reaction when I looked at Figure 2.5  was that actually there is quite a clear relationship between aggregate income levels and health/social problem outcomes in US states. Richer states have better outcomes even without special pleading for the influential peculiarities of Idaho, Wyoming, Montana....
At this point I began to lose a little confidence as you appear not to be  playing it straight with your readers, not all of whom, I assume, will look too closely at the figures or be  alert to what could be considered a bit of  textual leger de main. For what do you write about Figure 2.5 on page 21? I quote: "...Figure 2.5 shows that there is no clear relation between it [the Index of Health and Social Problems] and average income levels." (My emphasis). At the very least you have contradicted yourself within the space of 2 pages. Let's extend the benefit of the doubt: these things happen, we don't always correct our proofs accurately and quite innocently forget to make corrections when the time comes for reprinting. OK, I won't hold it against you as long as you will admit that my interpretation of your figures is admissible.
Figures 2.2-2.5 show the following: there is a quite strong correlation between the Index of Health and Social Problems and income inequality; there is a somewhat weaker correlation between the Index and per capita income. This is true whether we look at rich countries or US states (but especially true in the latter case). Unless income inequality and per-capita income are uncorrelated, which seems unlikely (especially in the country data if we always include the US) then inequality and income level are confounded at the aggregate level. That means that when we  look at the way in which inequality is related to the Index it partly captures the effect of income level and when we look at the "effect" of income level it partly captures the "effect" of inequality. Thus there is an illusion of naming. Inequality in your scatter plots actually means inequality plus a bit of per capita income. The only way to sort this out is to include both variables and look at the partial slopes. As a good Popperian I would go so far as to say that you should always include per capita income even if in these small samples its partial slope is "insignificant" (significance being just a matter of sample size) on the grounds that strong theories are the ones that pass the most stringent trials, not the ones that are given a bye in the first round. We should bend over backwards to test theories to destruction for it is only then that you can persuade the skeptics that actually care about the evidence - the ones that don't care are a lost cause and we shouldn't waste our time on them. That, in my view, is the difference between doing science and doing advocacy. But then you are both scientists and I scarcely need to tell you this.
But perhaps I shouldn't have got too worked up about such a small thing - twenty pages in and already nit-picking - after all in a scientific research programme nothing usually depends on just one study, one data-set or one data-point (let's face it, genuine critical cases are pretty hard to find). I felt  more comfortable when you continually reassured me that your conclusions were based on an enormous accumulation of evidence about which there was a clear scientific consensus.  Way to go! When the systematic reviews of all the relevant evidence consider data quality, effect size, scope conditions, sensitivity to robustness checks and so forth and then pronounce in your favour  "'tis a consummation devoutly to be wished for". We can tick that one off, pack our bags, go home and prepare for the next scientific challenge.
But I do want to read the systematic reviews myself and not just rely on being told what they say. Just as well really, for if I had to rely solely on the 456 citations in The Spirit Level I would never have come across the two lengthy  articles written by Lynch et al. in (2004) for the Milbank Quarterly entitled 'Is income inequality a determinant of population health?'. Luckily I have well informed colleagues who could point me in the right direction. Of course you know what Lynch et al. conclude from their careful scrutiny of "98 aggregate and multi-level studies examining the association between income inequality and health". For the benefit of those readers  who can't penetrate the pay-wall let me quote from their conclusions (pp 81): "Among affluent countries does income inequality help explain international differences in population health? The evidence suggests that income inequality is not associated with population health differences - at least not as a general phenomenon - among wealthy nations. Do levels of income inequality explain regional health differences within countries? In aggregate-level US studies, the extent of income inequality across states and metropolitan areas seems reasonably robustly associated with a variety of health outcomes, especially when measured at the state level. In multilevel US studies, using both individual and aggregate data, the evidence is more mixed, with state-level associations again being the most consistent. For other countries, the aggregate and multi-level evidence generally suggests little or nor effect of income inequality on health indicators in rich countries...but there may be some effects in the United Kingdom." [my emphasis].
Strange that you don't mention Lynch et al.'s papers (I know you cite them in your own review article but without, as far as I can see, any serious effort to explain why they get such different results to your own). Odd in several respects: firstly it was probably the most comprehensive independent (ie not counting papers written by yourselves) systematic review of the evidence on health then published when you were drafting The Spirit Level. Secondly, because it is not completely unfavourable to your position. After all it concludes that there is some evidence of an income inequality effect in US state level data (and possibly in the UK) - though their second paper which examines time-series data casts more doubt on the US case. I simply cannot understand why you fail to mention it, or if it is flawed in some way, rebut it, refer to your own rebuttal published elsewhere (if there is one) or the rebuttals of others (if there are any). Over and over again you tell us that the weight of the evidence is on your side and that there is a broad consensus amongst experts working in the field. But this simply isn't true, is it?  At the very least your now perplexed readers could be forgiven if they find your omission, well, a little shifty.
Still, let's not dwell on what isn't there and focus on what is. I was heartened to find in the 2010 reprinting of The Spirit Level a postscript dealing with the most important claims made by your critics. I want to take just one example from this with which I have some passing familiarity. On page 276 you start a section  Inequality, Class and Status. This caught my eye because it contains precisely one reference to indicate who or what you are replying to. It turns out that the object of your reply is a rather well known sociologist who happens to be a colleague and emeritus fellow of my college. I won't cause embarrassment by naming him here - anyone with a copy of The Spirit Level can easily look up the reference and make the identification if they care to. My colleague wrote a somewhat critical review of the book which was published in a well known  peer reviewed journal. After reading your "reply"  I was again deeply puzzled. What you say has, at most, only tangential relevance to the substance of my colleague's criticisms. The casual reader of what you write would come away with the impression that some sociologist had made a rather footling objection to the effect that you hadn't paid enough attention "to the vast amount of careful work now available on social class classifications" [your words] - surely a case of the cobbler only having eyes for leather. But wait a minute, that is not at all the substance of the critique.
Anyone who is able to read what my colleague actually wrote will realize that he has a much more fundamental objection to the income inequality hypothesis. Most won't be able to check though because the review is behind a pay-wall, so let me attempt a summary.
There is an incoherence at the heart of The Spirit Level. The micro-level explanation for the purported direct effect of aggregate income-inequality on individual level health is through a psycho-social mechanism couched in terms of a rather difficult to pin down notion of social status. This is strange and incoherent, because what  it amounts to is using income inequality as an indicator for status inequality. Why is it incoherent? Because it has been established in the social medicine literature that the health gradient with respect to social status is somewhat steeper then the health gradient with respect to social class and that income is more strongly related to social class than it is to status. Now at least some sociologists have a fairly clearly worked out idea of what the difference is between social class and social status. This is not the place to go into it, but perhaps an example will help. In Japan income inequality is less marked than in many developed nations and health outcomes are comparatively good. This would seem to conform to the Wilkinson-Pickett party line. But it is also the case that Japan is a highly status conscious (in the sociological sense) society. It is obligatory to acknowledge inferiority and superiority both in terms of behaviour and in terms of the use of honorifics. In any unfamiliar social situation the initial process of figuring out who is relatively inferior to whom is a source of considerable anxiety. To put it simply, in Japan  systematic inequality is strongly structured by considerations of social status, superiority and inferiority yet health outcomes are relatively favourable. If the relevant psycho-social mechanism is to do with social status (in any sociologically meaningful use of that term) then measuring status inequality by means of income-inequality puts Japan at the wrong end of the spectrum! This is much more than a petty point about occupational coding, but you wouldn't guess that if all you had to go on was The Spirit Level.
Well, by now if  you haven't already lost patience and dismissed me as yet another enemy of equality, you are probably muttering that you have dealt with all this before if only I would care to read more of your own work. The thing is, I have read it, and I'm not the only one to notice in it a recurrent pattern. Time after time you tell critics that you have dealt with their objection in one or another of your publications but when I turn to them what I find is indeed a reference to your critics, but not an actual response to the exact criticism they make  and often a discussion of some quite unrelated issue. Why you do this is, to me, quite baffling.
And disappointing, because the way I see it, we are on the same side of an equality debate that is much more important than whether there is or is not some substantively tiny direct effect on individual health outcomes of aggregate level income-inequality. Nobody disputes that individual level health disparities are related to differences (inequalities) in exposures to risk factors that are partly indicated  by (inter alia) individual level income differences. This in itself suggests that equalization of what in the Nordic welfare tradition would be called "the level of living" will likely have some effect on health disparities without requiring any commitment to a causal view about the direct effect of macro-level inequality which will,  in its turn, be affected by such an equalization. But, for what I speculate are quite understandable, but I think misguided, reasons, you want to go further than this. I think those reasons are ultimately to do with a political judgement. Like the old Fabians I think you believe that once the facts are discovered and disseminated to the public then  part of the political battle is won and people will persuade themselves to do the right thing. Thus it is important to you to demonstrate that even those at the top of the income hierarchy will benefit - have better health or higher levels of welfare in general - in counter factually more equal societies than the one they happen to live in. But you don't actually give us any strong reasons to believe this.  The non-linear shape of the standard welfare function already suggests that redistribution from rich to poor will make the rich a little worse off and the poor quite a bit better off and, setting aside disincentive effects, the average welfare level higher. In other words there is a pure efficiency argument for more equality which does not need to evoke either moral reasoning (which in my view gives other good grounds) or highly speculative psycho-social theories. So why should we believe that in the counter factual world implied by your model the rich will be better off (Why equality is better for everyone)?
There are in fact quite good reasons to believe that this is  unlikely.  This is just an intuition, but it is one that I think needs to be addressed. If there will be gains to the rich, how come they haven't spotted them and become the cheer leaders for more equality? Those at the top of the income distribution have been fairly adroit at spotting the main chances to perpetuate their relative social and economic positions. Should we now be asking: how come you are so rich when you are not smart enough to figure out that you would be even better off (in welfare terms) if you weren't quite as rich? And that raises the question of how the greater equality would be achieved. Redistribution towards equality with a fixed pot implies taking away income from those with more of it, or in the context of growth, reducing the increase in their income below what it otherwise would be. In both cases the rich will be experiencing a loss albeit in the second case a loss relative to a potential gain, yet The Spirit Level thesis implies that they should anticipate a welfare gain from this. If they don't then the implication is that they don't know what is good for them.
But how could they ever learn? The counter factual evidence generated by observational data is scarcely relevant. We can't draw the conclusion from cross-national comparisons that forcing rich Americans to live in Sweden will make them better off. The ceteris paribus clause just doesn't apply in reality. Whatever the numbers from the linear model say, it is a non assertable counter factual. The ceteris paribus clause would apply within their own country - at least for small changes at the margin - but now the relevant counter factual involves redistribution with all the apparent consequences discussed in the preceding paragraph and only applies when we contemplate small changes.
In my Old Labour way I tend to believe that very few political conflicts can be solved to everyone's satisfaction, because most serious conflicts are actually about ultimate values, rather than disputes about the technical means to reach agreed upon ends (other than vacuous Mom, apple pie and the American Way ends). Therefore trying to pretend that everyone is a winner is unlikely to be of much use. It would be better to put your bets on economic growth making a modest effort at redistribution less painful to those who are being relieved of potential but still tangible gains.
So, in the end Professors Wilkinson and Pickett, you face a credibility gap. People like myself who want, broadly speaking,  the same things as yourselves can find the time to ferret out, read and consider the evidence you don't tell us about. Joe Public, which I take it The Spirit Level is aimed at, has neither the time nor the access to the primary sources, let alone the training to make an informed judgement. They have to take what you say on trust. That is why university professors speaking with all the lustrous institutional prestige that implies have, in my opinion, a duty to be scrupulously honest, especially when writing for a popular audience. And when the brickbats come they should not be able to get away with emphasizing the popular nature of  their writing whilst ducking behind the protective shield of peer review.  What we all need are better  reasons to believe. I'm not the only social scientist or social democrat who thinks you  haven't yet given us nearly enough.

 Sincerely

Colin Mills


6 comments:

Anonymous said...

You didn't mention the educational value (although doubt the authors wish to be remembered for that). Even though it's been a few years, I still remember vividly the discussions we had on Wilkinson as part of the MSc course.

pe51ter said...

I enjoyed this article so will become a regular reader of thisd blog.

Benson Bear said...

I agree almost completely with what you have said here, except for two things.

First, your self-deprecating introduction seems inaccurate. You seem to have adequate credentials to expect a reply from Wilkinson had your comments been published in a place he considers appropriate, and I don't see any reason why they couldn't have been. (Perhaps this introduction was meant sarcastically as a jab at W, but that might further reduce your chance of a reply).

Second, you say "there is a pure efficiency argument for more equality which does not need to evoke either moral reasoning (which in my view gives other good grounds) or highly speculative psycho-social theories." Maybe there is such an argument, but it itself is a moral (broadly, consequentialist) argument, and indeed its moral force is weak. The less well off may benefit more from a redistribution, but that doesn't entail (especially in the minds of many better off) that they *deserve* it. It's obvious why W wants to use the argument he does: it gives the better off selfish-reason to act for more equality. Problem is, it is hard for him and others who want more equality on other grounds as well to retreat back to those grounds once his foray into "speculative psycho-social theories" is rebutted.

Anyway, what I would like to know is, since you wrote this, have you seen any kind of comments from W that address any of these points to any significant degree? I haven't found anything myself, and I think the fact that such things are hard to find may very well speak worse of him than do any flaws in his book or his previous work.

Unknown said...

Why not invite us to give a seminar in your department? We'd be happy to do so.

Colin said...

Kate, that is a very generous offer and I thing we would be delighted to take you up on it. Let's fix something up off-line.

Joseph Masters said...

As a member of Joe Public who has read the book and failed to think of any criticisms myself, I enjoyed this read. Thanks for helping me realise that the equality debate is slightly more complex than Pickett and Wilkinson make out.

Can I ask how the seminar went? Have there been any written rebuttals to the criticisms you make here?