It's been a bad week for my blood pressure. I'm trying manfully to ignore the hooplah arising from MT shuffling off her mortal coil, but it's more difficult to pretend that the BBC's Great British Class Survey (GBCS) didn't take place. In fact I should be feeling very guilty. Back in 2011 I promised that I would write something about it and didn't. Well now I will, but before I get going I'll reveal to you how I fared when I took the test. In fact, in the interest of science, I've taken it twice (test-retest reliability is a good thing to establish) first in 2011 and then just a couple of days ago. I can confirm that the test (at least in my case) has high reliability (my students will - if they've been listening to my lectures - wish to point out that reliability does not imply validity). On both occasions I scored in the high 90s (out of 100) on economic capital, and the low 20s on both social and cultural capital. I'll come back to this later, but on grounds of general ontological security I feel relieved to know that I'm roughly the same man I was 2 years ago. I also took a punt on the BBC's class calculator and I can proudly announce that it is official: I am a member of the Elite (despite appearing to have the social capital of a Trappist monk and the cultural capital of a Neanderthal). Well, I hope you are listening while I address you de haut en bas. I expect the knighthood is in the post.
Now to more serious matters. I want to ignore for the most part all of the media circus surrounding the story and do a deeply unfashionable and earnest thing: concentrate on the science. Behind all the fun and games is an article in the journal Sociology with the title: A New Model of Social Class: Findings from the BBC's Great British Class Survey Experiment (the title is actually slightly misleading because there are in fact no experimental findings - all the data is purely "observational"). Normally it would be behind a pay-wall but, for reasons that one can only speculate about, it is, at the time of writing, costless to view. This article, which has almost as many authors as the cast of a Cecil B. DeMille movie, is the source of the media frenzy. To keep things simple I'll refer to it as Savage et al. with the understanding that the unnamed others are also entitled to a fair share of the praise and the brickbats. We have to assume that it is a serious article, it is after all published in a peer reviewed official journal of the British Sociological Association. I'm going to do it the courtesy of taking it seriously in the belief that the authors would not wish for anything less.
Let's start with the praise. That can be disposed of pretty quickly. I think Savage et al. deserve a pat on the back for having the bright idea of getting the BBC to help them collect data. It was worth a try even if the results in terms of quality were utterly predictable. Of course the idea of using a mass media organization to collect data for scientific work is far from new. As far back as 1953/4 the BBC broadcast a programme called Science in the Making in which viewers were invited to participate in live TV experiments and fill in short questionnaires printed in the Radio Times. The data from at least one of these programmes, on right and left handedness, has survived and been used in academic papers.
The GBCS is, in effect, nothing more than a self-completion questionnaire administered over the internet. By their very nature self-completion questionnaires have to be pretty blunt in what they ask (at least if they are cheap). Respondents are left to their own devices to make what they can of the questions. Their great virtue is that you can collect large amounts of data, but this comes at the cost of losing control over who is actually providing the answers. All a respondent needs is a valid email address. Most people I know have more than one. As I let on above, I've contributed two cases. Somehow I doubt that most of the participants will have been enthusiastic about participating more than once, but strictly speaking there is no way to know that.
Let's stir in another ingredient that Savage et al. (to their credit) don't try to hide: there is massive self-selection bias in the GBCS. Surprisingly they demonstrate this with reference to the old, bad, inadequate NS-SEC categories that they want to distance themselves from. Managers and professionals are vastly over-represented whilst what one might speak of as the working-class are vastly under-represented. There is only a fifth as many routine and technical craft workers in the GBCS sample as there should be and only a third as many routine workers. With this level of sampling bias self-selection on unobservables becomes a serious problem. In short even after applying post-stratification weights the working class respondents, for instance, are likely to be atypical members of the working class (after all they have spontaneous chosen to fill in a BBC survey!) and can't give us unbiased information about the missing members of the target population. I want to be scrupulously fair: Savage et al. "fess up" and then pull off their master stroke. Despite having over 160,000 responses to their web survey almost all of the information from which they derive their empirical claims comes from a "nationally representative" quota sample of about a thousand respondents. Let me say that again, just in case you missed it: despite all the hullabaloo about the massive web based survey, almost everything the authors purport to find in their article is actually based on the flimsy basis of a quota sample n=1026. Without this their web based data is essentially useless. Given that the quota sample is the bedrock of the whole enterprise we might reasonably expect to be told something substantial about it. What quota controls were applied? Were they interlocking? What was the target sample? Which weights were applied and where did they come from? What was the mode of administration? Look carefully at the article: you won't find answers to any of these questions. Instead all we get is some vague hand waving (pp 6) to the effect that both GfK and the authors are satisfied that "the demographics are nationally representative". I'm not going to beat them up about the quota sample too much, except to point out one fairly obvious thing. With a probability sample one has a coherent set of procedures for making sample to population inferences including, most importantly, inferences about uncertainty: with a quota sample one has none, unless one can make a convincing argument that one has specified the "correct" data generating model. I doubt that the authors will wish to claim that they do the latter - it would, I think, be inconsistent with their general philosophy of data analysis which seems, to my eye, to favour more "exploratory" approaches. So Savage et al. have a problem. They have a mountain of highly self-selected data sitting on top of a mole hill of (slightly) better quality data. Ultimately they want to use the latter to make some sensible calibration of the former, but they have to do this without any sense of the uncertainty with which the calibration reference parameters have been estimated.
To make things concrete assume a simple random sample with n= 1000 and a characteristic that is found to be distributed 50/50 in the sample. Using the normal approximation to the binomial distribution gives us a 95% confidence interval for the population proportion of roughly 0.47-0.53. Now assume a characteristic that is distributed 6/94 in the sample ie roughly the estimated size of the "elite" class identified by Savage et al. The equivalent calculation gives a confidence interval of roughly 0.05-0.08. In practice these numbers are highly optimistic (they take no account of any source of uncertainty other than a highly specific type of sampling error ie measurement error, model uncertainty and non-sampling sources of uncertainty are ignored) and are but a rough and ready lower bound. Realistic uncertainty estimates will widen the intervals in some cases to a degree that is likely to be substantively important.
All of this sounds pretty daunting, but Savage et al. are in an even worse place. They don't have any coherent basis for making uncertainty estimates at all and, in fact, don't make any. Ignorance is truly bliss for some, but it doesn't make good science. The only way out would be to undertake a what if? experiment. In other words pretend that their quota sample was in fact generated by some known probabilistic mechanism and examine the implications of a range of plausible uncertainty estimates for key parameters on a hypothetical basis. This is really the only sensible way to even half convincingly calibrate the web survey data. I'd settle for that as making the best of a bad job.
That's enough, for the moment, about the data. I want now to turn to a more fundamental issue: how to think about concepts and typologies. Let's go back to Max Weber not because he is the fount of all truth and wisdom but because he had some rather specific and useful things to say about the matter. Concepts - like for instance social class - are abstract entities in the mind. We use them to selectively organize experience (on this point I could go back to Kant). We are free to organize experience on any basis we like and with any conceptual scheme we like, but there are consequences. Some conceptual schemes are of more practical use than others in the sense that they, as Linnaeus put it, "cleave nature at its joints". Typologies or categorization schemes are closely related to concepts, they are in a sense a step towards making the abstract idea more concrete. There can be as many typologies - and this is Weber's point - as there are points of view. This implies that coherent typologies are made for a purpose. Not everyone need have the same purpose and therefore emphasize the same things, but a purpose they must have. Normally that purpose is given by the part that a concept (and its associated typology) have in a theory that is meant to explain something and naturally this entails spelling out clearly what it is you want your concept to play a part in explaining. A typology is a practical tool that is to be used for a purpose and is not something that has any intrinsic value in and of itself. Inventing a typology for its own sake is a pretty pointless exercise.
So what is Savage et al.'s purpose? What do they expect their class typology to explain? Read the article as carefully as you like but you won't find an answer. There is a clue however in a pronouncement made by the lead author in a Guardian article:
"Let me be blunt. The concept of class matters, because we need a way of connecting accentuating economic inequalities to social and cultural differences which permeate our society. Rather than seeing our lifestyles and social networks as somehow separate from economic inequalities, there are overlaps that can work together to produce social advantage and disadvantage".
Blunt is definitely the right word here. Lifestyles, social networks, cultural differences and even economic inequalities are exactly what the Savage et al. typology cannot have any role in explaining, for the simple reason that all of these things are built into the definition of the typology. Whether these things "overlap" or not is completely besides the point and does not constitute a good reason for chucking them into the same food blender and then calling the mush that comes out of the other end "social class". If I want to examine the variation in the strength and depth of a population's social network ties - a perfectly reasonable thing to want to do - a tool that already is partly defined in terms of the explanandum - will not help me to discover any new knowledge. Quare Opium facit dormire: ... Quia est in eo Virtus dormitiva.
Nobody, it is true, has exclusive rights to the use of a word (or two). If Savage et al. wish to appropriate "social class" and fill it with their own content, so be it. But we are perfectly entitled to ask what precisely has been achieved by such an appropriation? Let's take a few steps back and consider one of the alternative usages.
Goldthorpe's most recent usage refers to aspects of the employment relationship - output/input monitoring difficulty and human asset specificity. I've played a small part in empirically examining the correspondence between these things and the occupational categories that form part of the NS-SEC. In Market, Class and Employment we go so far as to argue that in principle there is no reason why indicators of these things should not be measured at the level of the individual job and that if they were, then, in a sense, the categories of the NS-SEC would be redundant. In practice, of course, this is highly unrealistic. No general purpose survey is going to collect the detailed information required for accurate measurement of the real variables of interest and thus it makes perfect sense to chunk the world of employment up into occupations and then aggregate these into "classes" as long as we have evidence that these classes serve as adequate proxies for the underlying variables that practicalities dictate we can't measure directly. This is all really a matter of pragmatics disciplined by evidence and it would, of course, be absurd to attempt to "explain" say monitoring difficulty in terms of NS-SEC categories.
Savage et al. could say in reply that they were doing something very similar. Goldthorpe defines the content of class in terms of ideas drawn from Williamson's institutional economics, whilst they take their inspiration from across La Manche. For monitoring difficulty and asset specificity substitute economic capital, cultural capital and social capital and who is to say that this is not acceptable? Fair enough, we can take inspiration from wherever we like, but we then have to live with the consequences. The first thing we might think about is what is to be gained by throwing indicators of the three "capitals" into the blender. After all given the conventional definitions of these things we pretty much know what they refer to. Why muddy the waters by then throwing away information about variability along these three dimensions? - the inevitable consequence of any process that combines them to form a typology with a limited number of categories. As I often have occasion to say to my students: no, class is not income or education, we already have a good understanding of what the latter words mean, what in the world do we gain by calling them something else?
Goldthorpe can logically justify the "classification" of the underlying dimensions by saying, I'm creating a tool which can be used in situations and with data where there are no direct measures of the things I'm interested in. By well validated groupings of occupations I can capture a substantial part of what I mean by class - ie the employment relations relevant aspects of jobs. This move though is not open to Savage et al. The logic of their approach impels them to reject the pragmatics of occupational proxies and so makes it incumbent on them to provide a clear answer to the question: what exactly is gained from a data reduction exercise that reduces variation along three conceptual dimensions to a small set of categories?
Let's return to the theme of having a reason for creating categories. Goldthorpe had a clear reason for creating his class categories. He wanted to describe patterns of mass social class mobility in Britain and therefore he made distinctions that were sensible given that purpose. When that purpose changed - for example to describing cross-national patterns of mass social class mobility - further distinctions became necessary (farmers are numerically not that important in Britain but they are and were very important in France and Poland). Hence the evolution from the original class schema to the EGP schema and its variants (which Savage et al. completely confuse pp 3). Savage et al. perpetuate some highly fanciful ideas about the rationale behind the scheme. For example they tell us (pp 4) that:
"..it is increasingly apparent that a major appeal of the schema lay as a pragmatic means of placing individuals into social classes using standard nationally representative surveys with a moderate sample size. This is especially important since preferred forms of categorical data analysis such as log-linear modelling required reasonable cell sizes. It is for these pragmatic reasons, for instance, that an ‘elite’ was not distinguished (see Penn, 1981) within the Goldthorpe class schema."
This may be an accurate representation of what Penn believed but the attribution of a kind of technological determinism - log-linear modelling can't cope with small categories - as the primary reason for not distinguishing an elite is completely implausible. Recall, Goldthorpe was interested in mass mobility. The elite are irrelevant to his purposes and in any case in a sample of 10,000 men - the 1972 Oxford Mobility Survey - there would be so few of them sampled that any means of drawing highly accurate inferences about them - log-linear or otherwise - would be doomed to failure. There is a lesson to be learned here, if Savage et al. would care to learn it. It really helps to clarify your thinking when you have a point: what, ultimately, do you want to use your categories to explain?
It's time to move down the ladder of abstraction away from the big conceptual strategic issues and focus on issues of detail. Let's start with a quotation. In the course of discussing Goldthorpe's class schema Savage et al. say (pp 4):
"...the schema has been shown to be of less use in explicating the wider cultural and social activities and identities (see generally Devine, 1998; Savage,2000), which do not appear to be closely linked to people’s class position, as defined by the Goldthorpe class schema..."
This is an interesting claim, but I wonder how it sits with the evidence they set before our eyes. To make my point I have to reproduce Savage et al.'s Figures 1 and 2 (pp 9-10) and appeal to the principle of "fair usage". Figure 1 is a bi-plot derived from a multiple correspondence analysis of a whole bunch of categorical measures of the extent to which respondents in the small quota sample take part in various, broadly defined, cultural activities.
For those that care, a multiple correspondence analysis is just a simple correspondence analysis of a so called indicator matrix with respondents in the rows and dummy variable indicators of the responses chosen in the columns. In essence it is a data reduction technique which throws away vast amounts of information about higher order interactions between the variables.
Personally I find this a horrible graph created by people that obviously have little sympathy for the idea that the purposes of a graph is to convey a message to the reader as simply and clearly as possible - though to be fair, the pukka followers of Bourdieu do not set great store by clarity. Be that as it may, what do we learn? My first thought after puzzling over this for several minutes was that slightly greater clarity could have been achieved by a 45 degree rotation of all of the points in a clockwise direction, but nevertheless we are told that if we follow a line from NW to SE we can discern a dimension that puts high-brow cultural taste at the NW pole (people visit stately homes, go to the theatre, museums and galleries, dislike fast food etc) and the Royle family in the SE corner. Strange thinks I. If I look at Figure 2 which superimposes so called "supplementary points" onto Figure 1 what do I see?
What I see is that the categories of the old, bad, inadequate NS-SECs which Savage et al. believe "are not closely linked" to "cultural and social activities and identities" seem to fall from top to bottom roughly in a NW to SE fashion. What this means is that they are indeed predictive of the first cultural dimension of Figure 1. Personally I don't find this particularly surprising, but it beggars belief that Savage et al. didn't notice it. Did they really not see this or do they just not give a damn that their own evidence directly contradicts what they write? But what of the second dimension in Figure 1. This is apparently a great discovery and given the impressive label "emerging cultural capital" though in what sense it is emerging and what it is emerging from is never vouchsafed. This dimension runs from SW to NE and once you grasp the labels it is obvious that it distinguishes the sorts of things that young people like doing from the sorts of things that older people dislike or don't do.
This interpretation is confirmed by inspection of Figure 2. Fine. I don't doubt that younger people enjoy things that older people don't. My 6 year old has a great passion for moshi monsters that I don't share and you won't be able to get my mother-in-law down to the local Indian for a biryani, but what has that got to do with social class? Think about it further. Do people's tastes and activities change as they get older? Yes, they do. I haven't been down the gym or played five-aside football for a very long time nor have I recently been to see Coventry City play (all things I did when I was younger and more carefree). Do people's tastes depend on when they were born? Yes, I believe that is true too. My parents were well into their twenties and thirties when the sixties happened. They never really developed much of a taste for pop-music - Jim Reeves and Frank Ifield were more to my mum's taste. So what dimension 2 actually represents is a confounding of life-cycle and cohort effects. If we are to take it seriously as a measure of class we would be forced to believe that class can change simply as a function of getting older and being too stiff to get your football boots on or remains fixed because of the birth cohort you happen to be a member of. Both of these conclusions would be bizarre. Do Savage et al. really hold to them? I'm inclined to believe that they would deny they did, but they would seem to be implied by the logic of what they do. I'd be interested to know how they square this particular circle.
And finally we hit paydirt, if you've stayed with me so far you have probably guessed that I don't find A New Model of Social Class: Findings from the BBC's Great British Class Survey Experiment a very impressive piece of work. Its high profile in the media is, I think, not a positive thing for British sociology. In some domains people believe that any publicity is good publicity. In science, some sorts of publicity only serve to make the subject as well as its practitioners look ridiculous. That creates a negative externality for those of us who try, to the best of our abilities, to do decent science. That's why I will continue to post on conspicuous examples of bad sociology.
To make things concrete assume a simple random sample with n= 1000 and a characteristic that is found to be distributed 50/50 in the sample. Using the normal approximation to the binomial distribution gives us a 95% confidence interval for the population proportion of roughly 0.47-0.53. Now assume a characteristic that is distributed 6/94 in the sample ie roughly the estimated size of the "elite" class identified by Savage et al. The equivalent calculation gives a confidence interval of roughly 0.05-0.08. In practice these numbers are highly optimistic (they take no account of any source of uncertainty other than a highly specific type of sampling error ie measurement error, model uncertainty and non-sampling sources of uncertainty are ignored) and are but a rough and ready lower bound. Realistic uncertainty estimates will widen the intervals in some cases to a degree that is likely to be substantively important.
All of this sounds pretty daunting, but Savage et al. are in an even worse place. They don't have any coherent basis for making uncertainty estimates at all and, in fact, don't make any. Ignorance is truly bliss for some, but it doesn't make good science. The only way out would be to undertake a what if? experiment. In other words pretend that their quota sample was in fact generated by some known probabilistic mechanism and examine the implications of a range of plausible uncertainty estimates for key parameters on a hypothetical basis. This is really the only sensible way to even half convincingly calibrate the web survey data. I'd settle for that as making the best of a bad job.
That's enough, for the moment, about the data. I want now to turn to a more fundamental issue: how to think about concepts and typologies. Let's go back to Max Weber not because he is the fount of all truth and wisdom but because he had some rather specific and useful things to say about the matter. Concepts - like for instance social class - are abstract entities in the mind. We use them to selectively organize experience (on this point I could go back to Kant). We are free to organize experience on any basis we like and with any conceptual scheme we like, but there are consequences. Some conceptual schemes are of more practical use than others in the sense that they, as Linnaeus put it, "cleave nature at its joints". Typologies or categorization schemes are closely related to concepts, they are in a sense a step towards making the abstract idea more concrete. There can be as many typologies - and this is Weber's point - as there are points of view. This implies that coherent typologies are made for a purpose. Not everyone need have the same purpose and therefore emphasize the same things, but a purpose they must have. Normally that purpose is given by the part that a concept (and its associated typology) have in a theory that is meant to explain something and naturally this entails spelling out clearly what it is you want your concept to play a part in explaining. A typology is a practical tool that is to be used for a purpose and is not something that has any intrinsic value in and of itself. Inventing a typology for its own sake is a pretty pointless exercise.
So what is Savage et al.'s purpose? What do they expect their class typology to explain? Read the article as carefully as you like but you won't find an answer. There is a clue however in a pronouncement made by the lead author in a Guardian article:
"Let me be blunt. The concept of class matters, because we need a way of connecting accentuating economic inequalities to social and cultural differences which permeate our society. Rather than seeing our lifestyles and social networks as somehow separate from economic inequalities, there are overlaps that can work together to produce social advantage and disadvantage".
Blunt is definitely the right word here. Lifestyles, social networks, cultural differences and even economic inequalities are exactly what the Savage et al. typology cannot have any role in explaining, for the simple reason that all of these things are built into the definition of the typology. Whether these things "overlap" or not is completely besides the point and does not constitute a good reason for chucking them into the same food blender and then calling the mush that comes out of the other end "social class". If I want to examine the variation in the strength and depth of a population's social network ties - a perfectly reasonable thing to want to do - a tool that already is partly defined in terms of the explanandum - will not help me to discover any new knowledge. Quare Opium facit dormire: ... Quia est in eo Virtus dormitiva.
Nobody, it is true, has exclusive rights to the use of a word (or two). If Savage et al. wish to appropriate "social class" and fill it with their own content, so be it. But we are perfectly entitled to ask what precisely has been achieved by such an appropriation? Let's take a few steps back and consider one of the alternative usages.
Goldthorpe's most recent usage refers to aspects of the employment relationship - output/input monitoring difficulty and human asset specificity. I've played a small part in empirically examining the correspondence between these things and the occupational categories that form part of the NS-SEC. In Market, Class and Employment we go so far as to argue that in principle there is no reason why indicators of these things should not be measured at the level of the individual job and that if they were, then, in a sense, the categories of the NS-SEC would be redundant. In practice, of course, this is highly unrealistic. No general purpose survey is going to collect the detailed information required for accurate measurement of the real variables of interest and thus it makes perfect sense to chunk the world of employment up into occupations and then aggregate these into "classes" as long as we have evidence that these classes serve as adequate proxies for the underlying variables that practicalities dictate we can't measure directly. This is all really a matter of pragmatics disciplined by evidence and it would, of course, be absurd to attempt to "explain" say monitoring difficulty in terms of NS-SEC categories.
Savage et al. could say in reply that they were doing something very similar. Goldthorpe defines the content of class in terms of ideas drawn from Williamson's institutional economics, whilst they take their inspiration from across La Manche. For monitoring difficulty and asset specificity substitute economic capital, cultural capital and social capital and who is to say that this is not acceptable? Fair enough, we can take inspiration from wherever we like, but we then have to live with the consequences. The first thing we might think about is what is to be gained by throwing indicators of the three "capitals" into the blender. After all given the conventional definitions of these things we pretty much know what they refer to. Why muddy the waters by then throwing away information about variability along these three dimensions? - the inevitable consequence of any process that combines them to form a typology with a limited number of categories. As I often have occasion to say to my students: no, class is not income or education, we already have a good understanding of what the latter words mean, what in the world do we gain by calling them something else?
Goldthorpe can logically justify the "classification" of the underlying dimensions by saying, I'm creating a tool which can be used in situations and with data where there are no direct measures of the things I'm interested in. By well validated groupings of occupations I can capture a substantial part of what I mean by class - ie the employment relations relevant aspects of jobs. This move though is not open to Savage et al. The logic of their approach impels them to reject the pragmatics of occupational proxies and so makes it incumbent on them to provide a clear answer to the question: what exactly is gained from a data reduction exercise that reduces variation along three conceptual dimensions to a small set of categories?
Let's return to the theme of having a reason for creating categories. Goldthorpe had a clear reason for creating his class categories. He wanted to describe patterns of mass social class mobility in Britain and therefore he made distinctions that were sensible given that purpose. When that purpose changed - for example to describing cross-national patterns of mass social class mobility - further distinctions became necessary (farmers are numerically not that important in Britain but they are and were very important in France and Poland). Hence the evolution from the original class schema to the EGP schema and its variants (which Savage et al. completely confuse pp 3). Savage et al. perpetuate some highly fanciful ideas about the rationale behind the scheme. For example they tell us (pp 4) that:
"..it is increasingly apparent that a major appeal of the schema lay as a pragmatic means of placing individuals into social classes using standard nationally representative surveys with a moderate sample size. This is especially important since preferred forms of categorical data analysis such as log-linear modelling required reasonable cell sizes. It is for these pragmatic reasons, for instance, that an ‘elite’ was not distinguished (see Penn, 1981) within the Goldthorpe class schema."
This may be an accurate representation of what Penn believed but the attribution of a kind of technological determinism - log-linear modelling can't cope with small categories - as the primary reason for not distinguishing an elite is completely implausible. Recall, Goldthorpe was interested in mass mobility. The elite are irrelevant to his purposes and in any case in a sample of 10,000 men - the 1972 Oxford Mobility Survey - there would be so few of them sampled that any means of drawing highly accurate inferences about them - log-linear or otherwise - would be doomed to failure. There is a lesson to be learned here, if Savage et al. would care to learn it. It really helps to clarify your thinking when you have a point: what, ultimately, do you want to use your categories to explain?
It's time to move down the ladder of abstraction away from the big conceptual strategic issues and focus on issues of detail. Let's start with a quotation. In the course of discussing Goldthorpe's class schema Savage et al. say (pp 4):
"...the schema has been shown to be of less use in explicating the wider cultural and social activities and identities (see generally Devine, 1998; Savage,2000), which do not appear to be closely linked to people’s class position, as defined by the Goldthorpe class schema..."
This is an interesting claim, but I wonder how it sits with the evidence they set before our eyes. To make my point I have to reproduce Savage et al.'s Figures 1 and 2 (pp 9-10) and appeal to the principle of "fair usage". Figure 1 is a bi-plot derived from a multiple correspondence analysis of a whole bunch of categorical measures of the extent to which respondents in the small quota sample take part in various, broadly defined, cultural activities.
For those that care, a multiple correspondence analysis is just a simple correspondence analysis of a so called indicator matrix with respondents in the rows and dummy variable indicators of the responses chosen in the columns. In essence it is a data reduction technique which throws away vast amounts of information about higher order interactions between the variables.
Personally I find this a horrible graph created by people that obviously have little sympathy for the idea that the purposes of a graph is to convey a message to the reader as simply and clearly as possible - though to be fair, the pukka followers of Bourdieu do not set great store by clarity. Be that as it may, what do we learn? My first thought after puzzling over this for several minutes was that slightly greater clarity could have been achieved by a 45 degree rotation of all of the points in a clockwise direction, but nevertheless we are told that if we follow a line from NW to SE we can discern a dimension that puts high-brow cultural taste at the NW pole (people visit stately homes, go to the theatre, museums and galleries, dislike fast food etc) and the Royle family in the SE corner. Strange thinks I. If I look at Figure 2 which superimposes so called "supplementary points" onto Figure 1 what do I see?
What I see is that the categories of the old, bad, inadequate NS-SECs which Savage et al. believe "are not closely linked" to "cultural and social activities and identities" seem to fall from top to bottom roughly in a NW to SE fashion. What this means is that they are indeed predictive of the first cultural dimension of Figure 1. Personally I don't find this particularly surprising, but it beggars belief that Savage et al. didn't notice it. Did they really not see this or do they just not give a damn that their own evidence directly contradicts what they write? But what of the second dimension in Figure 1. This is apparently a great discovery and given the impressive label "emerging cultural capital" though in what sense it is emerging and what it is emerging from is never vouchsafed. This dimension runs from SW to NE and once you grasp the labels it is obvious that it distinguishes the sorts of things that young people like doing from the sorts of things that older people dislike or don't do.
This interpretation is confirmed by inspection of Figure 2. Fine. I don't doubt that younger people enjoy things that older people don't. My 6 year old has a great passion for moshi monsters that I don't share and you won't be able to get my mother-in-law down to the local Indian for a biryani, but what has that got to do with social class? Think about it further. Do people's tastes and activities change as they get older? Yes, they do. I haven't been down the gym or played five-aside football for a very long time nor have I recently been to see Coventry City play (all things I did when I was younger and more carefree). Do people's tastes depend on when they were born? Yes, I believe that is true too. My parents were well into their twenties and thirties when the sixties happened. They never really developed much of a taste for pop-music - Jim Reeves and Frank Ifield were more to my mum's taste. So what dimension 2 actually represents is a confounding of life-cycle and cohort effects. If we are to take it seriously as a measure of class we would be forced to believe that class can change simply as a function of getting older and being too stiff to get your football boots on or remains fixed because of the birth cohort you happen to be a member of. Both of these conclusions would be bizarre. Do Savage et al. really hold to them? I'm inclined to believe that they would deny they did, but they would seem to be implied by the logic of what they do. I'd be interested to know how they square this particular circle.
On the back of the multiple correspondence
analysis Savage et al. construct two summated ratings scales. The first
purports to measure the presence or absence of highbrow cultural capital the
second emerging cultural capital. One wonders why they went through all the
palaver. They could have just done a simple principal components analysis on
all their ratings scales and saved themselves a lot of trouble. It might even
have helped them get a clearer picture of their data for they could have given
the response categories plausible integer scores rather than having to work
with sparse data and categorical measures. Of course they would probably have
been summarily drummed out of the Bourdieu club for such a heresy and they would
have denied themselves the chance of using the elegant obfuscatory term
"modalities" which in plain Anglo-Saxon means "the levels of a
categorical variable". It also allows them to avoid any embarrassing
questions about reliability, alpha coefficients and so forth. It really would
have been nice to know how much variance was shared among the indicators
allocated to each of the dimensions. In the rarefied world of French influenced sociology such trivial
matters are clearly for the under-labourers rather than the Olympians
that boldly tread where none have gone before.
This seems like an appropriate time to go back to
the profile of scores I received on the three capital dimensions, for it raises some basic questions about the
validity of the measurement instruments themselves. If you recall, I revealed that I was
high on economic capital, but shamefully
low on cultural and social capital. I must say that I was puzzled by this. OK,
I exaggerate; most middle aged academics that live in a dual earning household
in the South-East of England
and work in a Russell group university are probably doing pretty well in income
terms. I know enough of the facts about the earnings and income distribution to
have a realistic idea of where my household sits in the pecking order. This is
why I cringe when I occasionally hear colleagues moaning about how badly they
are paid. I also have a house in the South-East with just a small mortgage on
it. That makes my housing wealth look pretty healthy compared to the national
average, though note gentle reader, the GBCS does not ask respondents to subtract
mortgage debt from the estimated value of their property.
What really threw me
was when I tried to answer the question about my savings. Though my liquid
assets are meager the GBCS instructed me to include the value of my pension.
This is a tricky calculation, and I doubt that most people bothered to attempt
it. To give even a moderately sensible answer you need to know what the cash
value of your pension will be at your retirement age, estimate how many years
you are going to live after retirement and deflate the sum by some appropriate
amount to get the present value. If you do all this and are optimistic about
your life chances this should augment the current net worth of a middle aged
person with a nice final salary occupational pension by a considerable amount.
Let’s just hope I live long enough to see any of it.
It seems to me that there
is plenty of scope for people to give silly answers to the savings questions.
What’s more it occurs to me when I look at Savage et al.’s Tables 6 and 7
(pp 12-13) that one of the things that may produce the apparent distinction they discover between
the “elite” and the “established middle class” is that the former are the
people who bothered to make a sensible calculation about their pension. Take a
look at the evidence in Table 6. Whereas the “elite” have salaries that are on
average less than twice as high as those of the “established middle class”
their savings are more than five times as large. They are also more than 10
years older (Table 7) and some of them may well actually be retired (the
average age is 57) and thus in a position to know what their pension is
actually worth. Another thing to notice is the average value of a member of the
elite’s house. From Table 6 we learn this is £325,000. Immediately I understand
why in one of the beautiful grey-scale maps that adorn the closing pages of the
article a large share of the “elite” live in London and the South-East. Living in a 3 bedroom
terraced house in a region with high housing
prices (as I do) would propel you quite a long way towards “elite” status
even if you were up to your neck in mortgage debt.
What then of my cultural capital? In my vainer
moments I think I’m actually quite a cultured bloke. I read a lot, in various
genres, classic novels, contemporary novels, history, philosophy, popular
science, current affairs even occasionally a bit of sociology. I watch a fair
number of art house films on DVD. I enjoy listening to Radio 4, jazz,
classical, opera, rock, folk, world and even country music. I live in a
bilingual household and I’ve spent extended periods of time living outside of
the UK.
By my reckoning that is a lot of cultural capital and we haven’t even got on to
my educational qualifications yet. But according to the GBCS I need to broaden
my horizons a bit and it isn’t just a matter of listening to a bit of hip-hop,
techno or heavy metal. Actually what I need to do is get out more. I don’t go
to the gym or watch live sports events at all, and for the past 6 years I’ve
rarely gone to a theatre, opera, art gallery, stately-home, museum or
restaurant. When I do go to the latter it tends to be either MacDonalds or
Pizza Express. Can you guess why? I’m sure you can. When you have young
children your life changes. If you eat out at all you eat where they want to
eat and you avoid going to places that they find boring. The average 6 year old
doesn’t like art-galleries, operas, stately-homes or museums. If you are a
reasonable parent, in your spare time you do stuff that your kids like. Before
I became a father I did all of those high-brow things and now, for the moment, I
don’t. Did I just lose all of my cultural capital (aka cultural resources) overnight?
It would be a funny kind of capital if that would be the case. I still, I
think, have the capacity to enjoy high-brow activities and it’s
an odd operationalization that makes the measurement dependent on life-cycle
stage. What I conclude from this is that with respect to cultural capital the
GBCS has a bit of a validity issue.
Moving right on to social capital, this is
supposed to be about the amount of contact that you have with other people and
the social range of those contacts. It’s a perfectly respectable idea, but I
don’t see that it has any straightforward connection to the idea of social
class. If you are a disadvantaged person I can see that having a broader rather
than a narrower social network may bring you some advantages. After all if you
are a domestic-cleaner and become unemployed
and you know a merchant banker they might employ you to clean their house or
look after their kids. It’s not obvious that if you are a merchant banker the
crude economic advantage of having a socially wider rather than a socially
narrower set of acquaintances is going to be that great. After all if you don’t
know someone who would love to look after your kids or clean your house you can always find somebody
from an agency or ask your next-door neighbour who they use.
In practice Savage
et al., while paying lip service to the notion of network diversity, just ignore
it and actually only use a measure of the average occupational status score of
the respondent’s social contacts and a count of what they call the “mean number
of social contacts reported”. The latter is actually inaccurate. It is in fact
just a count of the affirmative answers to a sequence of questions that ask: do
you know socially anyone who is a X? where X is any one of 34 different
occupations. So someone who knows a large number of stockbrokers and nobody else
would get a score of 1, as would somebody who has but a single friend in the
world. This is not a very sensible way to estimate the size of an ego-centric
network.
There is actually a very simple way to get an idea of the relative
size of the ego-centric networks in a population. You give people a list of
names taken from the page of a telephone directory or suchlike and you ask them
how many people they know who are called Smith, Jones, Johnson etc. Bizarre,
but probably inconsequential, is what Savage et al. have to say about the
measure of social status they use. This is the so called CAMSIS scale. Savage
et al. claim that it is a “widely validated” measure. This is an odd claim
because I know of no published or unpublished attempts to demonstrate the
criterion-related validity of this instrument. In fact I can report from
personal experience having a rather heated exchange with one of the original
constructors of the scale during which he both denied that it was either
possible or desirable to validate it in this sense just as it was impossible to
say what it was that the scale was actually a measure of! I was also present
when another of the creators of the scale waxed lyrical for an hour about his
special “theory of measurement” but alas proved unable to tell anyone what
exactly it was or how it differed from any other theory of measurement. But I
digress. Clearly my low social capital score owes something to being an
anti-social git, but it owes even more to the fact that those people I do know
come from a relatively narrow range of occupations. I know and socialize with
academics and professionals of various sorts. But that counts for little in the
GBCS. To improve my score I’ll need to get to know a much wider range of
people. Like a butterfly collector I should really go for one of each.
It’s time to move on to the piece-de-resistance –
the Savage et al. social classes themselves.
These fall out of the application of a particular statistical model – a
so called latent profile model – to the 1000 odd responses in the quota sample
plus 1 case consisting of the 161,400 responses to the GBCS each given a weight
of 1/161400. This means that the model estimates are derived almost entirely from
the small quota sample and the GBCS is basically discarded. Actually it is
quite a neat trick to rescue a slightly desperate situation. After all it would
have been embarrassing to have collected all that data and not used it.
It means that the GBCS data can be allocated to class categories even on the basis
of a model that is based almost entirely on the information contained in the
smaller sample.
It would be nice to know more about the latent
profile model, but we are given few details and this makes it
difficult to say anything sensible. I’m guessing that it makes some quite
strong assumptions – conditional independence of the observed variables given
the latent classes as well as marginal normality. The latter, if it is assumed,
seems quite implausible to me. I wonder how the referees of the paper managed
to make sense of the procedure, or were they just baffled by the apparent science? One
thing that is clear is the consequences of the model selection procedure. We
are told rather blandly that BIC is minimized when seven latent classes are assumed.
We are not told how much worse things are when 6 or 8 are assumed. In fact we
are given no sense whatsoever of model uncertainty or indeed why the gospel
according to BIC is to be preferred to any other model selection criterion. In
fact there is a dark secret hidden away here. The selection of seven classes,
much trumpeted in the media as the central finding of the study, is an artefact
of the sample size and without any well articulated theoretical grounds for
distinguishing the classes it is difficult to see how it could be otherwise.
It’s just as well that as things turned out, Savage et al. had, to all intents
and purposes, to discard the GBCS data. Imagine they had estimated their latent
profile model with 160,000 cases. The logic of their modus operandi would have
forced them to conclude that rather than there being seven classes there were
more like 77! The moral of the story is this: if you have no purpose in mind behind the application of a data reduction technique or no well articulated expectations, then the data will lead you by the nose. But "the data" or at least the amount of data is completely arbitrary and so will be your results.
The game is given away when Savage et al. attempt to interpret the seven social classes they "discover" (p 12 Tables 5 & 6). Normally one would do that in terms of the profiles of the observed variables associated with each of the latent classes. But looking, especially at Table 6, it becomes quickly apparent that this is rather difficult. Sure there are differences between the classes but which differences are important and which are trivial? There is no information presented here that would allow us to make a reasoned judgment using only the information that went into the latent profile analysis. In fact it is probable that the interpretation and almost certainly the labels applied to the classes actually comes from information quite extraneous to the latent profile analysis itself (pp 13 Table 7). Bizarrely one of the key ingredients is - wait for it - the percentage in or stemming from one of the old, bad inadequate, occupationally based NS-SEC categories! But Table 7 is quite revealing - actually of things that if Savage et al. were more Machiavellian they probably wouldn't have wanted to give away. Take for instance their "traditional working class". The average age of this group is 66! They have modest incomes, are low on "emerging cultural capital" yet 30% of them have, or had, jobs that would be classified as professional or managerial (don't you find that strange?). If the average age is 66 then a large proportion of this group are actually pensioners and may have nothing to do with the "working-class" traditional or otherwise. Pensioners are an important interest group, but what do we gain by calling them a social class? Life-cycle also I think plays a role in distinguishing what Savage et al. term the "elite" and the "established middle class". In terms of the variables tossed into the food blender the latter differ from the former principally because they have smaller incomes, less savings and live in cheaper houses. In terms of social and cultural capital they don't differ at all. When we look at Table 7 what we find is that the "established middle class" are on average 11 years younger than the elite and other differences are quite marginal. We can't tell how marginal because Table 7 gives us no way of calculating confidence intervals but a substantial part of the difference between these two classes could quite plausibly attributable to the life-cycle (plus not happening to live in the South-East).
It's getting time to wind up, but before I do, I want to bring up a point about the way Savage et al. set up their argument. On pages 4 and 5 of their article they motivate their intellectual project by listing five objections to the Goldthorpe class model. Fine, but what they say is either misleading, irrelevant or just plain false. Claim 1 is that the Goldthorpe scheme doesn't do well at predicting cultural consumption. Two points. Firstly it probably also doesn't predict well the colour of peoples' eyes, but who said a class scheme had to predict everything? Secondly, Savage et al.'s own data show that the NS-SECs do in fact predict cultural consumption! Claim 2 about the reasons for the "neglect" of the elite (are 6% really an elite?) I've dealt with above. Claim 3 is that economists have criticized the sociological way of doing social mobility using occupational categories and that income inequality within classes is growing. As far as I can tell these are false. It would be more accurate to say that economists have simply ignored the sociological way of doing social mobility and paid no attention whatsoever to any evidence not produced by an echt economist. So be it, I hold no brief for the way economists do their business. What is definitely not true however is the claim that in the UK within occupational earnings inequality has grown larger than between occupation inequality. If anything the latter has dominated - as has been carefully demonstrated by my ex doctoral student Mark Williams. Claim 4 is that "... a focus on occupations as the sole measure of class occludes the more complex ways that class operates symbolically and culturally, through forms of stigmatisation and marking of personhood and value." Maybe it does, but I can't tell because I don't really understand what that sentence actually means. What is clear though is that it functions in Savage et al.'s discourse in about the same way as saying bugaboo to a child does. Claim 5, at least in the way Savage et al. present it is simply absurd: "Daniel Oesch (2006) questions several of the assumptions in the EGP scheme for lacking the ability to take highly important horizontal cleavages into account, for giving a too homogenous description of the salaried middle class and for overdoing the manual/non-manual divide when separating between male production workers and routine sales and service occupations."
If Daniel Oesch does think this (and I've only got Savage et al.'s say so to go on) then he clearly hasn't understood the EGP class schema. What does he think the distinction between employees on the one hand and the self-employed, farmers and farm labourers on the other is if it is not a "horizontal cleavage"? And hasn't he noticed that in many applications Class IIIb is combined with Class VIIa? Just because a distinction exists it doesn't mean you have to use it. Back to my central theme of having a point.
And finally we hit paydirt, if you've stayed with me so far you have probably guessed that I don't find A New Model of Social Class: Findings from the BBC's Great British Class Survey Experiment a very impressive piece of work. Its high profile in the media is, I think, not a positive thing for British sociology. In some domains people believe that any publicity is good publicity. In science, some sorts of publicity only serve to make the subject as well as its practitioners look ridiculous. That creates a negative externality for those of us who try, to the best of our abilities, to do decent science. That's why I will continue to post on conspicuous examples of bad sociology.
3 comments:
Thanks for taking the trouble to write up such a detailed account! This post will serve as a useful point of reference (for all colleagues tempted by the GBCS). When I first read the results were out, I wondered how you/Oxford/Nuffield would react; after doing the "what class are you?" calculator and reading the abstract I wondered whether anyone would react at all (is it worth it to react?). Being an optimist, I point to the potential to interest students in class issues, and there's always educational value...
"what exactly is gained from a data reduction exercise that reduces variation along three conceptual dimensions to a small set of categories?"
Because its a very high profile part of a bigger, ongoing project to revive and/or maintain interest in class analysis?
i.e. it may well be methodologically pants, but there's a political backdrop here (which explains rather than justifies)?
S'funny the essentially arbitrary typology they come up with though given Professor Savage's (and his mate - whose good at counting - Andrew Miles) earlier wee gem of a book on the re-making of the working class and all that entails in terms of class consciousness i.e. how many people actually consider themselves/give a monkey's about "emergent service workers"? (something that could also be said about Goldthorpe's stuff, but then he was presumably asking a different question/has different interests)
Thank you for writing such an insightful piece. Your scientific analysis is very impressive. It is never easy to criticise a piece with huge publicity and approved publication in one of the top sociological journals. As you mentioned, the method of data collection and typology do not make any sense. It is more like a piece of work trying to shift the responsibility of social inequality from the government to individuals by addressing the importance of individual choice and taste in determining their social class rather than the unequal distribution of economic capital. I totally agree with your analysis and argument. Please keep writing on it.
Post a Comment